Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 26.11.12 in Vol 14, No 6 (2012): Nov-Dec

This paper is in the following e-collection/theme issue:

    Original Paper

    The Impact of Electronic Patient Portals on Patient Care: A Systematic Review of Controlled Trials

    1Institute of Health Informatics, UMIT - University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria

    2Institute of Public Health, Medical Decision Making and Health Technology Assessment, UMIT - University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria

    3Oncotyrol Center for Personalized Cancer Medicine, Innsbruck, Austria

    4Research Division for eHealth and Telemedicine, UMIT - University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria

    Corresponding Author:

    Alexander Hoerbst, PhD

    Research Division for eHealth and Telemedicine

    UMIT - University for Health Sciences, Medical Informatics and Technology

    Eduard-Wallnoefer-Zentrum 1

    Hall in Tyrol, 6060


    Phone: 43 (0)50 8648 ext 3814

    Fax:43 (0)508648673814



    Background: Modern information technology is changing and provides new challenges to health care. The emergence of the Internet and the electronic health record (EHR) has brought new opportunities for patients to play a more active role in his/her care. Although in many countries patients have the right to access their clinical information, access to clinical records electronically is not common. Patient portals consist of provider-tethered applications that allow patients to electronically access health information that are documented and managed by a health care institution. Although patient portals are already being implemented, it is still unclear in which ways these technologies can influence patient care.

    Objective: To systematically review the available evidence on the impact of electronic patient portals on patient care.

    Methods: A systematic search was conducted using PubMed and other sources to identify controlled experimental or quasi-experimental studies on the impact of patient portals that were published between 1990 and 2011. A total of 1,306 references from all the publication hits were screened, and 13 papers were retrieved for full text analysis.

    Results: We identified 5 papers presenting 4 distinct studies. There were no statistically significant changes between intervention and control group in the 2 randomized controlled trials investigating the effect of patient portals on health outcomes. Significant changes in the patient portal group, compared to a control group, could be observed for the following parameters: quicker decrease in office visit rates and slower increase in telephone contacts; increase in number of messages sent; changes of the medication regimen; and better adherence to treatment.

    Conclusions: The number of available controlled studies with regard to patient portals is low. Even when patient portals are often discussed as a way to empower patients and improve quality of care, there is insufficient evidence to support this assumption.

    J Med Internet Res 2012;14(6):e162





    The progress of modern information technology (IT) is changing and challenging health care. Clinical information systems as well as electronic health records have offered new opportunities for efficient and high-quality patient care [1].

    The emergence of the Internet and of the electronic health record (EHR) has brought new opportunities for a new and more active role of the patient [2-4]. The patient’s role is changing from a patronized patient to an informed patient and further to a responsible, autonomous and competent partner in his or her own care [5]. An active integration of the patient in his/her treatment bears multiple potential benefits such as fostering the quality of care as well as the compliance of the patient [6,7].

    One precondition for a more active patient’s role is to give the patient access to a providers’ documentation on previous or planned treatment. In many countries, patients have the right to access their clinical information whenever they request [8]. The Institute of Medicine argues that “patients should have unfettered access to their own medical information” and that this may help to increase quality of care and reduce medical errors [9]. However, patients demanding record access is not a common situation, due to cultural and practical reasons [10,11] and in part due to concerns by health care practitioners [8, 12]. Compared to paper-based solutions, information technological concepts such as electronic patient portals and personal health records (PHRs) seem to provide the opportunity to facilitate patients’ access to their clinical information.

    PHRs have been defined as “a set of computer-based tools that allow people to access and coordinate their lifelong health information and make appropriate parts of it available to those who need it” [13]. PHRs focus on functionalities where patients can document health-related data and can, if wanted, make them available to others, for example to their health care providers or families [14]. PHRs are typically owned and administered by the patients themselves.

    Electronic Patient Portals

    Electronic patient portals comprise provider-tethered applications that allow patients to electronically access health information that is documented and managed by a health care institution [15]. Patient portals are owned and administered by health care institutions (such as hospitals). As part of a patient portal, institutions may offer patients (typically web-based) access to selected clinical data that is governed by the respective institutions as part of the patients’ EHR. The patient can then access clinical data, read and print it, or integrate it into any (electronic or paper-based) type of patient-owned record. Besides providing access to EHR data, electronic patient portals may also offer additional services and functions to the patients. These include medication refills, appointment scheduling, access to general medical information such as guidelines, or secure messaging between a patient and an institution [15].

    In order to provide a distinctive definition for the current review, we define electronic patient portals as the class of applications provided and maintained by health care institutions that primarily allow access to clinical EHR data and secondarily may offer functions and services that are targeted towards enhancing medical treatment. For reasons of simplicity, all of these applications are called patient portals, regardless of their actual implementation e.g. as part of a PHR.

    The Impact of Electronic Patient Portals

    The idea of providing patients access to clinical information is not new. Traditionally, this has been done by providing a patient with paper-based copies of extracts from clinical documents [16-18]. Ross et al [8] have reviewed the effects of paper-based access to medical records and found that it has the potential for modest benefits for patient care, for example in enhancing doctor-patient communication, with only minimal risks such as increasing patient worry or confusion.

    In 2007, Ferreira et al [10] reviewed 14 papers that dealt with the effects of electronic patient portals on medical practice. He concluded that some studies indicated benefits of electronic patient portals, for example by enhancing communication, but that the studies also showed patients’ concerns about confidentiality and understandability of the content. The majority of the papers reviewed were surveys and interviews with patients and clinicians, without including a control group. Therefore, these studies were not able to objectively identify any benefits of patient portals when added to traditional patient care.

    To our knowledge, no systematic review of controlled studies on the impact of electronic patient portals has been conducted to date. With the further emergence of electronic patient portals in recent years, and the growing interest in evidence-based health informatics [19], we see the need to review the benefits of patient portals. The focus of this review is on the benefit of electronic patient portals for the patient, thus we focus on the impact of electronic patient portals on patient care. We did not predefine specific endpoints, but included all studies with endpoints, which were supposed to represent an impact on patient care by the study authors. For example, electronic patient portals may improve communication between provider and patient, or increase a patient’s adherence to medication treatment. Both could contribute to better patient care. Whether these benefits in fact arise when electronic patient portals are introduced needs to be shown by controlled trials and systematic reviews.


    The main objective of this paper is to systematically review the impact of electronic patient portals on patient care by analyzing controlled studies on the use of patient portals. The structure of this paper and the presentation of the results follow the PRISMA statement for reporting systematic reviews [20].


    Eligibility Criteria

    Papers were eligible if they presented controlled studies on the impact of patient portals. Impact could be visible in outcome-oriented parameters such as changes in mortality or morbidity or in costs of care. Also process parameters such as changes in therapy adherence or in patient satisfaction with the provided care were included, even when these parameters are merely surrogates for clinical outcome. With regard to study design, we included experimental (e.g. RCT) or quasi-experimental (e.g. controlled before-after trials) studies.

    In accordance with the definition given in the introduction for patient portals, patient portals are characterized by the following attributes:

    • electronic applications, typically web-based
    • provided and maintained by health care institutions
    • targeted towards providing functionality to all or a subgroup of patients
    • basic functionalities to access (a subset of) a patient’s clinical data
    • optional, additional functionalities such as communication modules, prescription refills, appointment scheduling, or educational guidelines

    The basic criterion for inclusion of a study was, however, that the application allowed the patient to access clinical data provided by a health care institution. We did not include papers that focused on telemonitoring systems, where the focus was on patients actively or passively providing data for their clinicians, or those that focused only on tailored messaging (e.g. of prevention information) from clinicians to patients such as Lin et al [21]. On the other hand, we included applications that were introduced as PHRs and looked for the functionality of an electronic patient portal, namely the possibility to access clinical information from a patient’s health care provider.

    We did not limit the search to a specific clinical setting, thus we included portals both in inpatient and outpatient areas. We included studies independent of the patient subgroup or disease (e.g. general portals, but also portals for diabetes patients).

    We limited the search to papers after 1990, as we did not expect to identify patient portal papers before that date. We excluded all papers where the intervention consisted of a paper-based copy of the medical record, as a systematic review on this topic has already been done [8]. We included papers in English, German and French.

    Information Sources

    We performed a literature search in April 2012, in PubMed Cochrane Library, CINAHL, EMBASE, ACM Digital Library and UMIT’s own Evaluation Database for relevant studies. Bibliographies of the included studies as well as of the reviews of Ross et al [8] and Ferreira et al [10] were used to identify additional studies.


    We used a combination of two queries (see Textbox 1). The first query searched for all papers about medical record systems that dealt with access to information or active patient participation. As the usage of MeSH headings was not consistent in all cases, we added a second query that looked at the term “patient portal” anywhere in the title or abstract.

    Textbox 1. Description of Queries.
    View this box

    After retrieving the results of the two queries, we imported all references to the Endnote reference manager and eliminated duplicates.

    Study Selection

    Two authors independently screened the titles and abstracts of all references to confirm whether the inclusion criteria were fulfilled. Differences were resolved by having a third author judge the paper. For all preselected papers, full text versions were retrieved and two authors independently determined whether the inclusion criteria were fulfilled. Differences were again resolved by consulting a third author.

    Data Collection Process

    Each included study was systematically described addressing clinical setting, type of intervention, type of study as well as outcome measures. Data extraction was done independently by two researchers. Results were compared and any differences were resolved by discussion.

    Data Items

    The following data items were documented for each study:

    • Time of study
    • Clinical setting
    • Type of patients included
    • Description of intervention
    • Description of control
    • Study design
    • Number of participants
    • Outcome measures
    • Study findings

    Assessment of Study Quality

    The study quality was assessed using the methodology checklists for RCT studies and for cohort studies of the National Institute for Health and Clinical Excellence (NICE) [22]. These checklists were applied independently by two reviewers (PSI, EA). Differences in judgment were solved by discussion.

    Synthesis of Results

    We systematically described the characteristics and results of the included studies in evidence tables. Further synthesis of results in the form of a meta-analysis was initially planned, but not possible due to the different outcome measures examined in the studies.

    Results were ordered according to the type of outcome measured, namely objective criteria and subjective criteria.


    Study Selection

    The queries in PubMed were performed in April 2012. Query 1 found 1,098 references, Query 2 found 52 references. Overall, when eliminating duplicates, we identified 1,136 references. We then checked Cochrane Library, CINAHL, EMBASE, ACM Digital Library and our own Evaluation Database at UMIT for relevant studies. We could not identify any further studies meeting our inclusion criteria. We also checked the reviews of Ross et al [8] and Ferreira et al [10] and citations in the included studies, but did not identify further studies.

    From the identified 1,136 papers, only 13 had an experimental or quasi-experimental study design (Multimedia Appendix 1). Of those 13 papers, 5 papers [23-27] presenting 5 distinct studies were finally eligible and then analyzed in detail. Two papers [26,27] describe the same study. Therefore, an overall total of 4 controlled studies were included in the review.

    Study Characteristics

    The 5 study papers presented evaluations of 4 different patient portals. Two papers [26,27] described different aspects of the same study. One portal was designed for patients undergoing IVF (in-vitro fertilization) treatment [23], one for diabetes mellitus patients [25], one for patients with congestive heart failure [26,27] and one was a general patient portal [24]. Three of the portals were located in the U.S. and one in the Netherlands. For details, see Table 1.

    Three of the studies were randomized controlled trials (RCT); one was a retrospective matched cohort study. The number of participants in each study ranged from 81 to 6,402 patients. The studies evaluated the impact on a variety of outcome criteria. One study focused on changes in clinical outcome parameters, including HbA1c, blood pressure, LDL, and medication adjustments [25]. One study focused on changes of resource consumption, including office visit rates and telephone contacts [24]. One study focused on changes of more subjective parameters such as patient satisfaction, patient knowledge, and patient anxiety [23]; these were measured by validated questionnaires. The fourth study combined changes of mortality, treatment adherence and resource consumption (message number) with subjective parameters (subjective health status, patient empowerment, medication adherence) [26,27].

    Impact of Patient Portals on Outcomes

    There were no statistically significant changes between intervention and control group in the 2 randomized controlled trials [25-27] investigating the effect of patient portals on endpoints measuring health or proxies for health (mortality, emergency room visits, hospitalizations, heart failure practice visits or risk factors). The use of patient portals showed no effect on all measurement scales to operationalize patient empowerment in one study [23].

    Statistically significant changes in the patient portal group, compared to a control group, could only be observed for the following parameters: quicker decrease in office visit rates and slower increase in the number of telephone contacts [24]; increase in number of messages sent [27]; changes of the medication regimen [24]; and better adherence to treatment [27]. For details on study design and measured outcomes, see Table 2.

    Table 1. Details of included studies – description of setting, intervention and control.
    View this table
    Table 2. Details of included studies – study design and outcome
    View this table

    Risk of Bias

    Overall risk of bias is unclear in 3 studies [24-27] and high for 1 study [23]. One RCT had no adequate concealment of allocation; for two RCTs the information was lacking. In the observational study, groups were matched for primary care physician, age, sex, and selected chronic conditions, but were potentially confounded by other factors eg, education cannot be excluded. Whether this could have led to a bias is unclear.

    In addition, in the 3 prospective studies, due to the nature of the intervention, patients and clinicians were not completely blinded to patients’ allocation. For example, as soon as the patient contacts the physician via secure messaging or brings along print-outs of the portal’s information, his or her allocation is known to the clinician. It is unclear, however, whether this may have led to a more intensive treatment of the patient and, if yes, whether this can be seen as a desired effect of portals, or as a possible source of bias. One study [23] additionally suffered from high drop-out rates. Here, too, the impact on the results is not clear.

    The studies used different methods and instruments to assess different types of impact. In most cases, the used questionnaires were based on validated survey instruments. Table 3 summarizes the risk of bias of each study. For details on each criterion, please see the NICE checklists for the respective RCT cohort studies [22].

    Table 3. Assessment of the quality of studies
    View this table


    We systematically searched the literature and found 4 controlled studies focusing on the impact of electronic patient portals. Given the fact that patient portals have been in use in the U.S. for several years, the number of controlled studies seems quite low.

    The studies were quite heterogeneous with regard to clinical setting, functionality of the intervention, and measured outcome. The different outcome parameters used made any further aggregation of results impossible, and showed how little evidence is available for each single outcome parameter. Most of the measured parameters did not show a statistically significant difference between intervention and control group. In particular, no statistically significant changes could be observed for parameters related to clinical outcome. Two studies found changes in contact patterns: quicker decrease in office visit rates and smaller increase in telephone contacts [24]; increase in number of messages sent [27]. Two studies found changes in medication regimen: higher changes of medication regimen [24]; better adherence to medication [27].

    Impact of Patient Portals

    We defined patient portals as presenting clinical information to the patients. Can we expect that giving patients access to clinical information can in general have an impact? The review of Ross et al [8] - who is, also the author of one of the studies we reviewed [27] - was dedicated to this question. It reviewed the outcome of 29 descriptive or controlled studies on adult patients’ access to (paper-based) medical records. Several studies showed an improvement of doctor-patient communication by patient-accessible medical records. There were, however, conflicting findings on improvements in treatment adherence, patient education, and patient empowerment; some controlled studies showed an improvement, while others did not. Ross et al [8] summarized that studies show potential for modest benefits, for example in enhancing doctor-patient communication, but that more research is necessary.

    Compared to paper-based access to records, electronic (web-based) patient portals allow a patient to access the information independently and repeatedly; the information is better legible; and the user can link the information to further sources of medical information available on the Internet [8]. Also, patient portals can be adapted to the patient’s wishes and knowledge level [23]. They can also be completed by secure communication links with health care providers or other functions. Overall, we could expect a higher impact of online portals compared to paper-based access. However, as our results show, the impact of patient portals, indicated by the studies reviewed here, is of a limited nature. In the following sections, we will discuss the findings of the 4 studies with regard to different topics.

    Impact on Clinical Outcome

    Grant et al [25] assessed changes in clinical parameters related to diabetes patients (such as HbA1, blood pressure, and LDL-C). He did not find statistically significant differences between both groups in general. But he found statistically significantly higher rates for medication adjustments of diabetes-mellitus related drugs. The portal they evaluated, however, included a module where a diabetes care plan was generated automatically based on the patient’s responses to short questions; these care plans may have led to the higher rate of medication adjustments in the intervention group, not so much the presentation of clinical data itself. In the study of Ross et al [27], mortality was compared between both groups and no differences could be seen. Overall, there are not sufficient studies to decide on the impact of patient portals on clinical outcomes.

    Impact on Health Resource Consumption

    Zhou et al [24] found a stronger decrease of annual primary care office visits in the intervention group compared to the control group when the intervention group used a patient portal with secure messaging. As explanation, Zhou mentions that a quarter of portal users indicated they would have scheduled an appointment in lieu of electronic messaging; so there seems to be a possibility of saving resources by a portal with electronic messaging. Ross et al [27] assessed a statistically significant increase of visits in the emergency room in the intervention group, but without temporal relationship between portal use and visits; also, no differences in hospitalization or visits to heart failure practice visits were observed. Thus, while the study of Zhou et al may indicate that a portal with electronic messaging may reduce the number of office visits, there is not sufficient data to decide conclusively on this.

    Impact on Patient Adherence

    Only the study of Ross et al [27] assessed adherence and found an increase in general adherence; this was measured by a validated questionnaire, not by objective data. Adherence to medication also increased, but did not reach statistical significance. In an earlier review, Ross et al [8] found 1 study with increased adherence, but 5 studies that could not support this. It seems plausible that better-informed patients show higher adherence to treatment or to clinical advices, but there is not sufficient evidence to support this assumption.

    Impact on Patient-Physician Communication

    Some of the reviewed studies addressed aspects of patient-physician communication. The general patient portal (KP HealthConnect) assessed by Zhou et al led to a slower increase of telephone contacts, and a quicker decrease of primary care office visits, compared to the control group [24]. In the study of Ross et al, the SPPARO portal for patients with congestive heart failure led to a statistically significant increase of the number of overall messages (electronic + phone) per patients [27], compared to the control group that just used phone. The authors argue here that SPPARO “appeared to supplement, rather than replace, telephone messages”. A consistent finding of these changes, or any related change in quality of communication, is not possible based on this data.

    Impact on Patient Empowerment

    Three of the reviewed studies addressed the concept of patient empowerment. The term “patient empowerment” has been controversially discussed in the literature [28], and a generally accepted definition seems to be missing [23, 29]. Consequently, each of the found studies used a different approach to measure patient empowerment.

    Tuil et al [23] measured a multidimensional concept composed of self-efficacy (using the General Self-Efficacy Scale [30]), actual and perceived knowledge, and patients’ involvement in the decision process. He did not find a statistically significant impact on any of those scales. Earnest et al [26] used a self-developed patient empowerment scale consisting of 8 questions (including feeling more in control, better prepared, feeling more reassured, trust, etc.); this study found a statistically significant decrease in patient empowerment scale scores over the study period in both groups, but no significant differences between both groups. Ross et al used the Kansas City Cardiomyopathy Questionnaire [31] to assess the health status of the participants; one subscale of it is devoted to self-efficacy. He found a trend (p=0.08) for an increase in self-efficacy in the intervention group, but with 6 points it was less than the predefined meaningful difference of 7.7 points. Overall, no study was able to show impact on patient empowerment. In their review of paper-based record access, Ross et al [8] also found some improvement in patient empowerment in randomized controlled trials, where patients felt “more in control” and “less helpless”; however, other controlled trials failed to support this finding. Overall, portals may have an impact on patient empowerment, but the evidence is not sufficient on this question.

    A review conducted by Samoocha [29] about the effectiveness of web-based interventions on patient empowerment arrives at the same conclusion. There are disease-specific self-efficacy effects that could be found, but a general increase in self-efficacy could not be identified, as evidence is not sufficient.

    Impact on Patient Satisfaction

    Tuil et al [23] used the Patient Satisfaction Questionnaire [32] and found no differences between both groups. Ross et al [27] assessed patient satisfaction with the Art of Medicine Questionnaire and found improvements in two questions: How well patients felt their problems were understood, and how well doctors explained information. For the other four questions, no impact was seen. After adjustment for multiple testing, there was no statistically significant effect at all. The review of paper-based access of Ross et al [8] found 6 studies focusing on this topic, none of them showed statistically significant differences. Overall, access to information is probably only one facet of patient satisfaction; it is therefore questionable whether the impact of a patient portal on patient satisfaction is measurable.

    Meaning and Generalizability of Findings

    The results presented by the 4 studies did not contain convincing evidence for a general positive impact of electronic patient portals on clinical outcome, resource consumption, patient satisfaction or other variables compared to conventional ways of communication. Three of 4 studies were conducted in the U.S. The generalizability to health care settings in other countries is unclear.

    Outcome research regarding patient portals is still at its beginning, and most of the analyzed studies could not show clear benefits for the patient regarding quality of patient care. Given the large resources needed to build and maintain patient portals, health care institutions should carefully weigh costs and (expected) benefits.

    There may be several explanations for the missing evidence of the benefit of patient portals:

    Electronic portals provide information from the medical record to patients. However, better-informed patients are not necessarily healthier patients [27, 33]. Descriptive evidence from a large number of studies suggests that patients are interested in access to their patient records, and that they find it helpful and useful [10, 34-36]. These findings, however, do not guarantee that there is in fact a measurable impact on health, as a better-informed patient is only one (possibly minor) factor contributing to the quality of care.

    Studies in which a patient portal was combined with further services, such as secure messaging, interactive decision-support or health-related reminders, showed more positive impact on patient outcomes, patient-provider communication, disease management, and patient satisfaction, as a recent review of diabetes portals showed [37]. The interactive guiding and coaching of patients may be more effective than purely presenting clinical information without further advice.

    Especially patients with chronic diseases (for example, with diabetes mellitus, congestive heart failure) and patients with intensive and long-time treatment (for example, IVF) may be more willing to use electronic portals [27]. Nevertheless, these groups may be already actively communicating with their physicians, therefore a portal does not show additional impact. This could explain that the studies in our review (with 3 of 4 studies focusing on these types of patients) did not show statistically significant impact.

    Finally, only a minority of patients may be interested in using patient portals. Less computer literate, less motivated or less ill patients may not be interested. For example, in the Kaiser Permanente Northwest region (Oregon and southwest Washington), only 6% of all members have registered to the patient portal [24], and Weingart reports [38] a 11% utilization rate among primary care patients. Some study authors report difficulties in recruiting participants for the study [25], and some found that study participants are typically higher educated and have higher income than non-participants [27]. All this leads to the question whether patient portals may increase the digital divide, an issue also discussed by others [38].


    Quality of the Studies

    Three studies had sample sizes with less than 200 patients. Only one (retrospective) study included more than 6,000 patients. There was one study with a high risk of bias and no study with a low risk of bias. Crucial criteria to assess the risk of study bias, such as randomization method, concealment of allocation or blinding, were not reported in all publications.

    None of the studies gave clear information as to how often the participants in the study group used the portal. Only the authors of the SPPARO study [26,27] mentioned that 80% of participants used the portal at least once, with a median of eight days during a one-year study period, which correlated roughly to the number of office visits during this time. The authors concluded that the patients did not use the portal repeatedly between these visits. In the other studies, no information on usage patterns is given; therefore, it is not clear whether the patients really exploited the offered functionalities. This may have reduced the measurable impact of the portals.

    All electronic patient portals included in this review, offered functionality in addition to sole access clinical data. This renders comparison of studies difficult. In addition, it makes it difficult to identify to which functionality (clinical data access or additional functions) the measured effect can be attributed.

    Completeness of the Review

    We conducted a systematic literature search, but may have overlooked studies that were unpublished or in the grey literature.

    With 4 identified studies, the available evidence is quite limited. Despite a comprehensive query, we cannot be sure to have identified all related studies, as the terms used in the title, abstract, and keywords are not uniformly used. For example, the concept “patient portal” has been circumscribed in the literature as shared medical record, access to medical record, online PHR, or online medical record.

    It can be questioned how the benefit of patient portals can be operationalized at all – did the studies use the correct operators (e.g. number of phone contacts)? In the future, more research seems necessary on meaningful indicators that measure the effects of patient portals, and more patient portals should undergo systematic evaluation studies.


    Even if electronic patient portals are often seen as a way to empower patients and improve patient care, the available evidence does not support this assumption. Further studies of larger sample size and with harmonized outcome indicators are needed to investigate this question.


    We thank the reviewers for their detailed and helpful comments on the first version of the manuscript.

    This work was partly supported by the COMET Center Oncotyrol which is funded by the Austrian Federal Ministries of BMVIT/BMWFJ (via FFG) and the Tiroler Zukunftsstiftung/Standortagentur Tirol (SAT).

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Excluded papers.

    XLS File (Microsoft Excel File), 220KB


    1. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006 May 16;144(10):742-752. [Medline]
    2. Munir S, Boaden R. Patient empowerment and the electronic health record. Stud Health Technol Inform 2001;84(Pt 1):663-665. [Medline]
    3. Joubert M, Gaudinat A, Boyer C, Geissbuhler A, Fieschi M, Hon Foundation Council Members. WRAPIN: a tool for patient empowerment within EHR. Stud Health Technol Inform 2007;129(Pt 1):147-151. [Medline]
    4. Hoerbst A, Kohl CD, Knaup P, Ammenwerth E. Attitudes and behaviors related to the introduction of electronic health records among Austrian and German citizens. Int J Med Inform 2010 Feb;79(2):81-89. [CrossRef] [Medline]
    5. Nagel G. Patientenkompetenz. Krankenhauspharmazie 2005;26:128-133.
    6. Sittig DF. Personal health records on the internet: a snapshot of the pioneers at the end of the 20th Century. Int J Med Inform 2002 Apr;65(1):1-6. [Medline]
    7. Detmer D, Bloomrosen M, Raymond B, Tang P. Integrated personal health records: transformative tools for consumer-centric care. BMC Med Inform Decis Mak 2008;8:45 [FREE Full text] [CrossRef] [Medline]
    8. Ross SE, Lin CT. The effects of promoting patient access to medical records: a review. J Am Med Inform Assoc 2003 Apr;10(2):129-138 [FREE Full text] [Medline]
    9. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington: National Academy Press; 2001.
    10. Ferreira A, Correia A, Silva A, Corte A, Pinto A, Saavedra A, et al. Why facilitate patient access to medical records. Stud Health Technol Inform 2007;127:77-90. [Medline]
    11. Hackl W, Hoerbst A, Ammenwerth E. The electronic health record in Austria: physicians' acceptance is influenced by negative emotions. Stud Health Technol Inform 2009;150:140-144. [Medline]
    12. Hackl WO, Hoerbst A, Ammenwerth E. "Why the hell do we need electronic health records?". EHR acceptance among physicians in private practice in Austria: a qualitative study. Methods Inf Med 2011;50(1):53-61. [CrossRef] [Medline]
    13. Markle Foundation. Connecting for Health - A Public-Private Collaborative. 2003. The Personal Health Working Group - Final Report   URL: [accessed 2012-11-19] [WebCite Cache]
    14. Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs). J Am Med Inform Assoc 2008 Dec;15(6):729-736 [FREE Full text] [CrossRef] [Medline]
    15. Bourgeois FC, Mandl KD, Shaw D, Flemming D, Nigrin DJ. Mychildren's: integration of a personally controlled health record with a tethered patient portal for a pediatric and adolescent population. AMIA Annu Symp Proc 2009;2009:65-69 [FREE Full text] [Medline]
    16. White P. Copying referral letters to patients: prepare for change. Patient Educ Couns 2004 Aug;54(2):159-161. [CrossRef] [Medline]
    17. Sheldon MG. Giving patients a copy of their computer medical record. J R Coll Gen Pract 1982 Feb;32(235):80-86 [FREE Full text] [Medline]
    18. Drury M, Yudkin P, Harcourt J, Fitzpatrick R, Jones L, Alcock C, et al. Patients with cancer holding their own records: a randomised controlled trial. Br J Gen Pract 2000 Feb;50(451):105-110 [FREE Full text] [Medline]
    19. Ammenwerth E, de Keizer N. A viewpoint on evidence-based health informatics, based on a pilot survey on evaluation studies in health care informatics. J Am Med Inform Assoc 2007 Jun;14(3):368-371 [FREE Full text] [CrossRef] [Medline]
    20. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009;6(7).
    21. Lin CT, Wittevrongel L, Moore L, Beaty BL, Ross SE. An Internet-based patient-provider communication system: randomized controlled trial. J Med Internet Res 2005;7(4):e47 [FREE Full text] [CrossRef] [Medline]
    22. NICE. The Guideline Manual 2009 - Methodology checklist. London, UK: National Institute for Health and Clinical Excellence; 2009.
    23. Tuil WS, Verhaak CM, Braat DD, de Vries Robbé PF, Kremer JA. Empowering patients undergoing in vitro fertilization by providing Internet access to medical data. Fertil Steril 2007 Aug;88(2):361-368. [CrossRef] [Medline]
    24. Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care 2007 Jul;13(7):418-424 [FREE Full text] [Medline]
    25. Grant RW, Wald JS, Schnipper JL, Gandhi TK, Poon EG, Orav EJ, et al. Practice-linked online personal health records for type 2 diabetes mellitus: a randomized controlled trial. Arch Intern Med 2008 Sep 8;168(16):1776-1782. [CrossRef] [Medline]
    26. Earnest MA, Ross SE, Wittevrongel L, Moore LA, Lin CT. Use of a patient-accessible electronic medical record in a practice for congestive heart failure: patient and physician experiences. J Am Med Inform Assoc 2004 Oct;11(5):410-417 [FREE Full text] [CrossRef] [Medline]
    27. Ross SE, Moore LA, Earnest MA, Wittevrongel L, Lin CT. Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004 May 14;6(2):e12 [FREE Full text] [CrossRef] [Medline]
    28. Powers S. Empowerment as treatment and the role of health professionals. ANS Adv Nurs Sci 2003;26(3):227-237.
    29. Samoocha D, Bruinvels DJ, Elbers NA, Anema JR, van der Beek AJ. Effectiveness of web-based interventions on patient empowerment: a systematic review and meta-analysis. J Med Internet Res 2010;12(2):e23 [FREE Full text] [CrossRef] [Medline]
    30. Schwarzer R, Jerusalem M. Generalized Self-Efficacy scale. In: Weinman J, Wright S, Johnston M. editors. Measures in health psychology: a user's portfolio. Windsor: NFER-Nelson; 1995:35-37.
    31. Green CP, Porter CB, Bresnahan DR, Spertus JA. Development and evaluation of the Kansas City Cardiomyopathy Questionnaire: a new health status measure for heart failure. J Am Coll Cardiol 2000 Apr;35(5):1245-1255. [Medline]
    32. Grogan S, Conner M, Norman P, Willits D, Porter I. Validation of a questionnaire measuring patient satisfaction with general practitioner services. Qual Health Care 2000 Dec;9(4):210-215 [FREE Full text] [Medline]
    33. Noell J, Glasgow RE. Interactive technology applications for behavioral counseling: issues and opportunities for health care settings. Am J Prev Med 1999 Nov;17(4):269-274. [Medline]
    34. Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006 Apr;13(2):121-126 [FREE Full text] [CrossRef] [Medline]
    35. Hassol A, Walker JM, Kidder D, Rokita K, Young D, Pierdon S, et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc 2004 Dec;11(6):505-513 [FREE Full text] [CrossRef] [Medline]
    36. Delbanco T, Walker J, Bell SK, Darer JD, Elmore JG, Farag N, et al. Inviting Patients to Read Their Doctors' Notes: A Quasi-experimental Study and a Look Ahead. Ann Intern Med 2012 Oct 2;157(7):461-470. [CrossRef] [Medline]
    37. Osborn CY, Mayberry LS, Mulvaney SA, Hess R. Patient web portals to improve diabetes outcomes: a systematic review. Curr Diab Rep 2010 Dec;10(6):422-435 [FREE Full text] [CrossRef] [Medline]
    38. Weingart SN, Rind D, Tofias Z, Sands DZ. Who uses the patient internet portal? The PatientSite experience. J Am Med Inform Assoc 2006 Feb;13(1):91-95 [FREE Full text] [CrossRef] [Medline]


    DM: Diabetes Mellitus
    EHR: Electronic Health Record
    ICSI: Intracytoplasmic Sperm Injection
    IVF: In-Vitro Fertilization
    NICE: National Institute for Health and Clinical Excellence
    PHR: Personal Health Record
    RCT: Randomized Controlled Trial
    SPPARO: System Providing Patients Access to Records Online

    Edited by G Eysenbach; submitted 25.06.12; peer-reviewed by A Moen, P Knaup-Gregori, U Hübner, B Crotty, D Firth; comments to author 16.07.12; revised version received 01.10.12; accepted 16.10.12; published 26.11.12

    ©Elske Ammenwerth, Petra Schnell-Inderst, Alexander Hoerbst. Originally published in the Journal of Medical Internet Research (, 26.11.2012.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.