Preprints (earlier versions) of this paper are available at, first published .
Validation of a Novel Electronic Health Record Patient Portal Advance Care Planning Delivery System

Validation of a Novel Electronic Health Record Patient Portal Advance Care Planning Delivery System

Validation of a Novel Electronic Health Record Patient Portal Advance Care Planning Delivery System

Original Paper

1Division of General Internal Medicine, Department of Internal Medicine, College of Medicine, The Ohio State University, Columbus, OH, United States

2Medical Student Research Program, College of Medicine, The Ohio State University, Columbus, OH, United States

3College of Public Health, The Ohio State University, Columbus, OH, United States

Corresponding Author:

Seuli Bose-Brill, MD

Division of General Internal Medicine, Department of Internal Medicine

College of Medicine

The Ohio State University

895 Yard Street

Columbus, OH, 43212

United States

Phone: 1 614 293 7980


Background: Advance care planning allows patients to articulate their future care preferences should they no longer be able to make decisions on their own. Early advance care planning in outpatient settings provides benefits such as less aggressive care and fewer hospitalizations, yet it is underutilized due to barriers such as provider time constraints and communication complexity. Novel methods, such as patient portals, provide a unique opportunity to conduct advance care planning previsit planning for outpatient care. This follow-up to our pilot study aimed to conduct pragmatic testing of a novel electronic health record-tethered framework and its effects on advance care planning delivery in a real-world primary care setting.

Objective: Our intervention tested a previsit advance care planning workflow centered around a framework sent via secure electronic health record-linked patient portal in a real-world clinical setting. The primary objective of this study was to determine its impact on frequency and quality of advance care planning documentation.

Methods: We conducted a pragmatic trial including 2 sister clinical sites, one site implementing the intervention and the other continuing standard care. A total of 419 patients aged between 50 and 93 years with active portal accounts received intervention (n=200) or standard care (n=219). Chart review analyzed the presence of advance care planning and its quality and was graded with previously established scoring criteria based on advance care planning best practice guidelines from multiple nations.

Results: A total of 19.5% (39/200) of patients who received previsit planning responded to the framework. We found that the intervention site had statistically significant improvement in new advance care planning documentation rates (P<.01) and quality (P<.01) among all eligible patients. Advance care planning documentation rates increased by 105% (19/39 to 39/39) and quality improved among all patients who engaged in the previsit planning framework (n=39). Among eligible patients aged between 50 and 60 years at the intervention site, advance care planning documentation rates increased by 37% (27/96 to 37/96). Advance care planning documentation rates increased 34% among high users (27/67 to 36/67).

Conclusions: Advance care planning previsit planning using a secure electronic health record-supported patient portal framework yielded improvement in the presence of advance care planning documentation, with highest improvement in active patient portal users and patients aged between 50 and 60 years. Targeted previsit patient portal advance care planning delivery in these populations can potentially improve the quality of care in these populations.

J Med Internet Res 2018;20(6):e208




Advance care planning (ACP) is the formal process of outlining a patient’s future care preferences should they lose the ability to make informed decisions for themselves [1,2]. ACP documentation provides guidance in accordance with patient care preferences to proxy decision makers and medical teams in times of medical crisis. Patients with documented ACP experience increased adherence to their desired medical preferences [3], higher rates of palliative management, fewer hospitalizations, and increased quality of life near death [1]. Similar studies have found that patients with a plan for future medical care spend less time in the hospital during their last year of life and have their wishes more frequently respected by family members [1,4-6]. ACP also has the potential to reduce expensive health care interventions not wanted by the patient, such as lengthy critical care stays at the end of life [7]. Despite these benefits, the Center for Disease Control and Prevention estimates that ACP completion rates are around 30%, even with advancements in the electronic health record (EHR) and ACP delivery [8].

The Institute of Medicine recommends conducting ACP early in a patient’s chronic disease diagnosis, with periodic reassessment every several years or with change in prognosis (such as new diagnosis, hospitalization, or worsening of chronic disease) [9]. This ACP communication is best provided in the primary care setting [10]; yet, primary care remains ill-equipped to systematically conduct ACP discussions due to the competing care demands and fast pace of appointments [11]. As few as 1% of Medicare beneficiaries with an established primary care physician report having an ACP conversation with their health care provider [12]. Identified barriers include provider time constraints, uncertain patient prognosis, emotional complexity of ACP decisions, and difficulty in information sharing within and between health organizations [13]. The Centers for Medicare and Medicaid have recognized the need for early ACP by providing reimbursement for team-based primary care ACP discussions occurring under physician supervision [14]. These current gaps in care, juxtaposed with the urgent need for effective outpatient ACP care models, require development and rapid dissemination of innovative ACP strategies [15].

EHR-linked patient portals, first described in the 1990s and now ubiquitous due to EHR Meaningful Use guidelines [16], allow patients to electronically communicate with their medical providers within a secure platform. Patient portal communication has driven innovation in chronic disease management, population management, and previsit planning with strategies such as incorporating a patient portal refill button for hypertension patients; sharing top-priority problem list information with complex diabetes patients; and providing influenza vaccine outreach. These strategies allow providers to more effectively use time during an appointment by providing preparatory communication to patients before the appointment begins [17-21]. Patient portal–based outreach for health maintenance has been reported to marginally improve health maintenance behaviors such as flu vaccination (by 1-2%) [17]. Response rates to provider-initiated patient portal communication for chronic disease management have been reported at about 15% [22]. Patient portals are not without their own unique barriers; one study found that patients chose not to activate their portal due to a lack of sufficient instructions, privacy concerns, preference for face-to face interaction, or connectivity obstacles [23-26]. Despite these limitations, patients and care teams are increasingly actively incorporating portals into medical management.

Framework and Objectives

In an earlier study, our research team developed and pilot-tested a concise EHR patient portal–linked, electronic ACP communication framework in a small randomized controlled trial [11,27]. The framework was developed through incorporating best practice guidelines, and then refined through focus group feedback and cognitive interviewing [11]. This developed framework consists of an introduction to ACP and key evidence-based questions that can be sent to patients for response outside of their office visit. By allowing patients to think and comment about their future wishes for care in advance of visits, we hoped to maximize patient-provider time in the office visit for advanced communication and documentation. The framework responses were automatically stored in a patient’s medical record for retrieval by clinical staff or physicians at office visits. Patient responses to the framework could also be sent to the primary care provider for review [11]. Use of this framework in a small pilot study demonstrated improvement in ACP documentation rates and quality [11]. Even though our framework was the first to be piloted in the field [11], its feasibility and impact on outcomes in a real-world primary care setting when integrated into actual previsit planning algorithms in the course of actual clinical primary care were yet to be determined [28].

The aim of this study was to determine the impact of previsit ACP planning using a secure EHR-linked framework upon ACP documentation when incorporated into a real-world primary care environment.

This study was approved by the Ohio State University Institutional Review Board.


Patients 50 years or older, presenting for a preventive health or chronic disease follow-up visit, with an active MyChart account, at a participating clinical site were included in the study. Patients did not need prior MyChart experience. There were 2 clinical sites participating in the study. Sites were selected based on their demographic, size, and provider similarity, as well as their uniform clinical practices with respect to ACP delivery. Each clinical site used the same ACP practices before the study period, which included an institutional packet of information on ACP, state-issued documents about Advance Directives (ADs), and encouragement to discuss any ACP questions with their provider. The usual care site maintained these practices throughout the duration of the study. The intervention site incorporated an open-ended ACP framework (containing 4 questions), sent via a patient’s EHR-tethered patient portal, into a clinical practice algorithm. Physicians, nurses, and other clinical staff at the intervention clinic were collectively involved in developing this ACP previsit planning algorithm that was rolled out practice wide over a 3 month period (Figure 1). The algorithm focused on promoting patient/provider communication surrounding ACP preferences, rather than intervening specifically on completed ADs scanned into the EHR. Furthermore, 219 patients in the control group and 200 patients in the intervention group were cared for during the study period, yielding 419 total participants. Further demographic characteristics are outlined in Table 1 for both the control and intervention sites. This study measured the impact of a clinical intervention rolled out practice wide. Our study team received a Waiver of Consent by the Institutional Review Board to assess the pragmatic impact of this clinical intervention. It was not possible to blind this study because chart review automatically revealed participants who received the ACP framework versus those who did not. We mitigated the inability to blind this study by using binomial metrics, such as documentation present/absent, and rigid scoring criteria for quality. We also conducted spot checking of reviews to ensure accuracy of dataset, as outlined above.

Participants did not know their intervention was the intervention of interest. Each clinical site agreed to participate in an ACP process study. However, providers and patient participants did not receive labels about whether they were receiving intervention or usual care. Practice workflow was implemented without labels.

Data security was paramount in this study. We used clinical staff (Institutional Review Board approved) routinely interacting with the patient record and completing previsit planning for clinical care to administer the intervention. Our research team was embedded in the clinical site. Data were housed within the secure institutional firewall and only accessed within the clinical site. Only de-identified datasets were shared with the statistical team for analysis using a secure, institutional drive. The delivery system developed by the practice providers had built-in safeguards for addressing clinical emergencies, such as patients responding to the secure message with medical complaints, by having a nurse and physician on call for urgent messages.


Charts were reviewed both 1 week and 4 weeks post appointment. Charts were reviewed by a member of the team who had received training and quality control checks on ACP chart review protocol. Chart review findings were spot checked by a second member of the team (one every 20 records) to ensure accuracy. The protocol outlined that training and education interventions would be used to respond to discrepancies in chart review rates and quality assessments. However, interventions were not needed because spot checking did not yield discrepancies. The participant’s demographics, presence of ACP (including before and after the visit), quality of ACP if present, and number of MyChart messages sent in the last year were recorded. The intervention charts were reviewed to see if the patient had read the intervention on their portal and responded to any of the questions.

Figure 1. Intervention workflow that was implemented at the study practice during the 3-month trial period. ACP: advance care planning; PCP: primary care physician.
View this figure
Table 1. Demographic information from both the control and intervention clinic.
Demographic informationControl (n=219)Intervention (n=200)
Median age, years6161
Age range, years50-9350-91
Male, n (%)76 (35)101 (50.5)
Female, n (%)143 (65)99 (49.5)
Number of chronic diseasesa44
Number of medicationsa76

aMedian number per patient at each practice site.

Quality of ACP was measured using a 20-point scoring criteria entitled “Criteria for Scoring Quality of ACP Documentation” (Multimedia Appendix 1) that has been used in previous studies [11,27]. The rubric was created by our team after reviewing best practice reports on how to measure ACP quality, such as those from the United Kingdom National Health Service, the Australian Quality Advance Care Planning Board, and the National Hospice and Palliative Care Organization’s “Caring Connections” program [29-31]. This method of scoring the quality of ACP has not been validated, but assigned points for the presence of items in the patient’s ACP. The ability to quantify the quality of our patient’s ACP was a crucial component of the study and gave us more thorough feedback on the utility of the novel framework.


Our study analyzed the documentation rates and quality of ACP across both study sites, especially focusing on new ACP documentation appearing in the EHR. To assess whether or not the increase in new documentation was significant between the 2 sites, a Fisher exact test was used. To analyze quality, a Mann-Whitney test was used to test the significance in new ACP quality between the 2 sites. The data were also analyzed by age and portal usage. Participants were separated into age groups by decade and portal usage was defined as either high or low, with high usage being more than 10 portal messages in 1 year.


Of the 200 patients who were sent the intervention, 156 read the message (78.0% read rate) on their portal and 39 responded (19.5% response rate) to at least one question in the framework (see Multimedia Appendix 2). Of those who responded to our intervention, 49% (19/39) already had some form of ACP documented in their EHR and 51% (20/39) added ACP to their EHR for the first time, yielding a 105% (19/39 to 39/39) increase in ACP documentation rates. Responders with existing ACP had a mean quality score of 4.94, as compared with a mean score of 4.09 for all documented ACP at the intervention arm during our study period. For respondents without prior documented ACP, the intervention alone yielded a mean quality score of 3.7. Respondents sent a median 11 MyChart messages per year and had a median age of 63 years. MyChart usage did not increase due to the intervention at the site overall; patients had a median 5 messages per year at our intervention arm before and during the study period.

Table 2. Documented advance care planning (ACP) in electronic health record. Documentation rates represent the percentage of charts that had any form of ACP, and quality is rated by the 20-point scoring criteria.
Patient characteristicControlIntervention
All patients

ACPa documentation rate, N219200

Preintervention, n (%)129 (58.9)74 (37.0)

Postintervention, n (%)130 (59.3)94 (47.0)

Rate percentage increase, n (%)0.727.0

Quality of all documented ACP, N13094

Quality rating postintervention, mean3.264.09
Patients aged 50-60 years

ACP documentation rate, N10996

Preintervention, n (%)54 (49.5)27 (28)

Postintervention, n (%)55 (50.4)37 (39)

Rate percentage increase, n (%)1.837

Quality of all documented ACP, N5537

Quality rating postintervention, mean2.813.75
Patients who are high portal users (>10 messages in 1 year)

ACP documentation rate, N8267

Preintervention, n (%)51 (62)27 (40)

Postintervention, n (%)51 (62)36 (54)

Rate percentage increase, n (%)033

Quality of ACP documentation, N5136

Quality rating postintervention, mean3.254.19

aACP: advance care planning.

Our intervention did not appear to affect the percentage of patients who had a scanned document in their EHR; both before and after the intervention, approximately 14% (28/200 and 7/47) of patients had a scanned directive at that practice. One patient brought in an Advance Directive to be scanned after responding to our framework.

Documentation and Quality Rates

ACP documentation in the EHR increased by 27.0% (74/200 to 94/200) during the study period at our intervention site, compared with a 0.7% (129/219 to 130/219) increase at our control site (Table 2; see Multimedia Appendix 1 for scoring criteria used for rating quality). A Fisher exact test was used to determine the significance of the differing increase in new documentation rates during the study period and yielded P<.001, indicating that patients exposed to our intervention were more likely to document ACP than those receiving usual care. A Mann-Whitney test was used to see if new ACP documented under our intervention was higher in quality and yielded P<.001, indicating that having the intervention led to a statistically significant ACP quality difference.


Patients aged between 50 and 60 years saw the greatest increase in ACP completion rates. At our intervention site, documentation rose 37% (27/96 to 37/96) as compared with 1.8% (54/109 to 55/109) at the control site. Comparatively, the 61-70 age group saw a 31% (29/76 to 38/76) increase in documentation rates, and the 71-80 age group saw a 6% (17/26 to 18/26) increase at the intervention site. Our control site had a 0% increase in each of those 2 age groups, but higher baseline rates of ACP completion before the study period (64% and 81%). In the intervention arm, there was only 1 patient between 80 and 89 years, and 1 patient over 90 years, so there were insufficient data to analyze this group. Individuals in the 50-60 age group, however, had slightly lower ACP quality as compared with the study population as a whole.

MyChart Users

Those who sent more than 10 MyChart messages in 1 year were defined as “High Portal Users” and comprised approximately a third of the study group at each site. Documentation rose by over 33% (27/67 to 36/67) at the intervention site for this group. Comparatively, low portal users (10 messages or less) at our intervention site saw a 23.4% (47/133 to 58/133) increase in documentation rates.

Patient Applications

In this study, we found that patients exposed to our framework were significantly more likely to have ACP documentation in the EHR and the quality of that documentation was better. This intervention benefits both the patient and the provider by providing another way for patients to think through the difficult decisions of how they envision their future care before their office visits. For patients who already used the patient portal, adding the framework would be a seamless integration into their usual part of their care. This tool was used most frequently by patients in the 50-60 age group and already active on MyChart. Targeting patients who are high users or in this age demographic to receive this tool can be a strategy for providing high-yield individualized previsit planning for ACP using patient portals. This intervention did not capture many different demographics, including all nonportal users, so developing other strategies to improve ACP documentation against cultural, technological, and demographic barriers must be used to ensure that there are improved outcomes for all and to continue to address existing health disparities in ACP documentation [32]. Previous research has found that patients who are middle-aged, male, and have greater disease burden are more likely to use their patient portals, which is similar to the trend we found in our study [33]. Few studies report on response rates to practice level, provider-initiated patient portal interventions; however, response rates to our framework (20%) using this previsit planning system were higher than response rates to a similar intervention for primary care depression screening and management (15.4%) using a secure patient portal messaging system [22].

Office Workflow Applications

In terms of workflow, the framework requires a member of the care team to send out the MyChart message 3-5 days before the appointment. If a patient responded, answers were appropriately documented and sent to the patient’s provider. In our study, the messages were sent out by a clinic nurse who could also answer any follow-up questions the patient had and then route the message to the appropriate provider. With increased team-based, patient-centered medical home patient outreach before appointments in primary care settings, as well as ubiquitous use of patient portals for practices to adhere to meaningful use guidelines, these interventions can be disseminated to a wide array of primary care practices. Higher rates of ACP documentation resulted, while reducing time needed to have a complete ACP discussion with the patient during the office visit, as existing answers have already been recorded and the patient had preparatory time to articulate their wishes.

Long-Term Clinical Applications

Previous studies have shown the benefits of previsit planning; if the provider has documentation of some of the patient’s future care preferences beforehand, there can be a more productive discussion with the patient during their appointment [34]. As end-of-life discussions can be difficult for patients, having a standardized framework that is sent to all patients over 50 years helps normalize the discussion and better prepare the patient [34]. Physicians can also tailor their future ACP discussions based off the patient’s documented answers from the framework, allowing them to be more effective conducting individualized ACP discussions [35]. For provider workflow, having the ACP process begin before the appointment can save time during the appointment and documentation afterwards, helping to ensure communication about end-of-life preferences occurs despite competing priorities [36].

The study was not designed to elicit qualitative feedback from patients and providers to promote its pragmatic implementation. However, the participating site liked the delivery system enough to implement it as a permanent intervention. Furthermore, the participating site shared the intervention, which has now been disseminated to the wider net of associated primary care sites at the institution.


Baseline rates of ACP documentation at each site were different, as noted in the results section, with the usual care site having higher rates of completed ACP documentation at baseline. However, preintervention chart review at both sites allowed assessment of typical documentation rates, to determine the change in documentation rates in intervention versus control before and after the intervention period. Additionally, patients had to have an activated MyChart account to be included, which excluded a portion of the clinic population.


Incorporating the patient portal into ACP delivery is a promising way to increase completion rates and efficiently facilitate the conversation between the provider and the patient about their future wishes. This strategy may be more effective in patients familiar with patient portal use, who regularly use patient portal communication to access clinical care.


The project described was supported in part by the OSU College of Medicine Roessler Research Scholarship (MF).

Authors' Contributions

SB-B, MF, LM, LP, AC, and SK were involved in the concept and design; SB-B, MF, LM, and AC were involved in the acquisition of data; SB-B, MF, LM, and LP were involved in the analysis and interpretation of data; SB-B, MF, LM, and SK were involved in the drafting of the manuscript; SB-B, MF, LM, LP, AC, and SK were involved in the critical revision of the manuscript for important intellectual content; MF and LP were involved in the statistical analysis; and SB-B was involved in supervision.

Conflicts of Interest

SB-B was involved in developing the tested framework that is now available for nonproprietary use. She was not involved in administering the intervention or completing the primary chart review.

Multimedia Appendix 1

Advance Care Planning Quality Grading. Criteria for scoring quality of ACP documentation in the EHR.

PDF File (Adobe PDF File), 25KB

Multimedia Appendix 2

Framework Questionnaire.

PDF File (Adobe PDF File), 28KB

  1. Mullick A, Martin J, Sallnow L. An introduction to advance care planning in practice. BMJ 2013 Oct 21;347:f6064. [Medline]
  2. Brinkman-Stoppelenburg A, Rietjens JA, van der Heide A. The effects of advance care planning on end-of-life care: a systematic review. Palliat Med 2014 Sep;28(8):1000-1025. [CrossRef] [Medline]
  3. Hammes BJ, Rooney BL, Gundrum JD. A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem. J Am Geriatr Soc 2010 Jul;58(7):1249-1255. [CrossRef] [Medline]
  4. Wright AA, Zhang B, Ray A, Mack JW, Trice E, Balboni T, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA 2008 Oct 08;300(14):1665-1673 [FREE Full text] [CrossRef] [Medline]
  5. O'Malley AJ, Caudry DJ, Grabowski DC. Predictors of nursing home residents' time to hospitalization. Health Serv Res 2011 Feb;46(1 Pt 1):82-104 [FREE Full text] [CrossRef] [Medline]
  6. Molloy DW, Guyatt GH, Russo R, Goeree R, O'Brien BJ, Bédard M, et al. Systematic implementation of an advance directive program in nursing homes: a randomized controlled trial. JAMA 2000 Mar 15;283(11):1437-1444. [Medline]
  7. Khandelwal N, Kross EK, Engelberg RA, Coe NB, Long AC, Curtis JR. Estimating the effect of palliative care interventions and advance care planning on ICU utilization: a systematic review. Crit Care Med 2015 May;43(5):1102-1111 [FREE Full text] [CrossRef] [Medline]
  8. Koeppen E, Burkard J, Ward E. Improving Advance Directive Completion Rates through Advance Care Planning. Doctor of Nursing Practice Final Manuscripts 2017:57 [FREE Full text]
  9. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, D.C: The National Academies Press; 2015.
  10. Ramsaroop SD, Reid MC, Adelman RD. Completing an advance directive in the primary care setting: what do we need for success? J Am Geriatr Soc 2007 Feb;55(2):277-283. [CrossRef] [Medline]
  11. Bose-Brill S, Kretovics M, Ballenger T, Modan G, Lai A, Belanger L, et al. Testing of a tethered personal health record framework for early end-of-life discussions. Am J Manag Care 2016 Jul 01;22(7):e258-e263 [FREE Full text] [Medline]
  12. Keary S, Moorman SM. Patient-Physician End-of-Life Discussions in the Routine Care of Medicare Beneficiaries. J Aging Health 2015 Sep;27(6):983-1002. [CrossRef] [Medline]
  13. Bernacki RE, Block SD, American College of Physicians High Value Care Task Force. Communication about serious illness care goals: a review and synthesis of best practices. JAMA Intern Med 2014 Dec;174(12):1994-2003. [CrossRef] [Medline]
  14. Centers for Medicare and Medicaid Services. 2015. Advance Care Planning   URL: https:/​/www.​​Outreach-and-Education/​Medicare-Learning-Network-MLN/​MLNProducts/​Downloads/​AdvanceCarePlanning.​pdf [accessed 2018-02-26] [WebCite Cache]
  15. Lund S, Richardson A, May C. Barriers to advance care planning at the end of life: an explanatory systematic review of implementation studies. PLoS One 2015 Feb 13;10(2):e0116629 [FREE Full text] [CrossRef] [Medline]
  16. Centers for Disease Control and Prevention. 2017. Meaningful Use   URL: [accessed 2018-02-26] [WebCite Cache]
  17. Cutrona SL, Sreedhara M, Goff SL, Fisher LD, Preusse P, Jackson M, et al. Improving Rates of Influenza Vaccination Through Electronic Health Record Portal Messages, Interactive Voice Recognition Calls and Patient-Enabled Electronic Health Record Updates: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2016 May 06;5(2):e56 [FREE Full text] [CrossRef] [Medline]
  18. Grant RW, Wald JS, Poon EG, Schnipper JL, Gandhi TK, Volk LA, et al. Design and implementation of a web-based patient portal linked to an ambulatory care electronic health record: patient gateway for diabetes collaborative care. Diabetes Technol Ther 2006 Oct;8(5):576-586 [FREE Full text] [CrossRef] [Medline]
  19. Irizarry T, DeVito Dabbs A, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res 2015 Jun 23;17(6):e148 [FREE Full text] [CrossRef] [Medline]
  20. Grant RW, Uratsu CS, Estacio KR, Altschuler A, Kim E, Fireman B, et al. Pre-Visit Prioritization for complex patients with diabetes: Randomized trial design and implementation within an integrated health care system. Contemp Clin Trials 2016 Mar;47:196-201 [FREE Full text] [CrossRef] [Medline]
  21. Sarkar U, Lyles CR, Parker MM, Allen J, Nguyen R, Moffet HH, et al. Use of the refill function through an online patient portal is associated with improved adherence to statins in an integrated health system. Med Care 2014 Mar;52(3):194-201 [FREE Full text] [CrossRef] [Medline]
  22. Pecina J, North F, Williams MD, Angstman KB. Use of an on-line patient portal in a depression collaborative care management program. J Affect Disord 2017 Jan 15;208:1-5. [CrossRef] [Medline]
  23. Goel MS, Brown TL, Williams A, Cooper AJ, Hasnain-Wynia R, Baker DW. Patient reported barriers to enrolling in a patient portal. J Am Med Inform Assoc 2011 Dec;18(Suppl 1):i8-12 [FREE Full text] [CrossRef] [Medline]
  24. Ronda MC, Dijkhorst-Oei LT, Rutten GE. Reasons and barriers for using a patient portal: survey among patients with diabetes mellitus. J Med Internet Res 2014 Nov 25;16(11):e263 [FREE Full text] [CrossRef] [Medline]
  25. Tieu L, Sarkar U, Schillinger D, Ralston JD, Ratanawongsa N, Pasick R, et al. Barriers and Facilitators to Online Portal Use Among Patients and Caregivers in a Safety Net Health Care System: A Qualitative Study. J Med Internet Res 2015 Dec 03;17(12):e275 [FREE Full text] [CrossRef] [Medline]
  26. Uhler LM, Pérez Figueroa RE, Dickson M, McCullagh L, Kushniruk A, Monkman H, et al. InformedTogether: Usability Evaluation of a Web-Based Decision Aid to Facilitate Shared Advance Care Planning for Severe Chronic Obstructive Pulmonary Disease. JMIR Hum Factors 2015 Feb 25;2(1):e2 [FREE Full text] [CrossRef] [Medline]
  27. Bose-Brill S, Kretovics M, Ballenger T, Modan G, Lai A, Belanger L, et al. Development of a tethered personal health record framework for early end-of-life discussions. Am J Manag Care 2016 Jun;22(6):412-418 [FREE Full text] [Medline]
  28. Chalkidou K, Tunis S, Whicher D, Fowler R, Zwarenstein M. The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clin Trials 2012 Aug;9(4):436-446. [CrossRef] [Medline]
  29. Australian Government Department of Health and Ageing. Canberra: Commonwealth of Australia; 2010. A National framework for advance care directives   URL: [accessed 2018-02-26] [WebCite Cache]
  30. Henry C, Seymour J. NCPC. 2007. Advance care planning: a guide for health and social care staff   URL: [accessed 2018-02-26] [WebCite Cache]
  31. CaringInfo, National Hospice and Palliative Care Organization. Advance Care Planning   URL: [accessed 2018-02-26] [WebCite Cache]
  32. Li IF, Hsiung Y. Exploring Advance Care Planning in Taiwanese Indigenous Cancer Survivors: Proposal for a Pilot Case-Control Study. JMIR Res Protoc 2017 Dec 21;6(12):e238 [FREE Full text] [CrossRef] [Medline]
  33. Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013 Nov 19;159(10):677-687. [CrossRef] [Medline]
  34. Au DH, Udris EM, Engelberg RA, Diehr PH, Bryson CL, Reinke LF, et al. A randomized trial to improve communication about end-of-life care among patients with COPD. Chest 2012 Mar;141(3):726-735 [FREE Full text] [CrossRef] [Medline]
  35. Modan G, Brill SB. Engaging death: Narrative and constructed dialogue in Advance Care Planning discussions. Commun Med 2014;11(2):153-165. [Medline]
  36. Sinsky CA, Sinsky TA, Rajcevich E. Putting Pre-Visit Planning Into Practice. Fam Pract Manag 2015;22(6):34-38 [FREE Full text] [Medline]

ACP: advance care planning
EHR: electronic health record

Edited by G Eysenbach; submitted 17.10.17; peer-reviewed by K Powell, T Irizarry; comments to author 07.12.17; revised version received 27.02.18; accepted 14.03.18; published 26.06.18


©Seuli Bose-Brill, Michelle Feeney, Laura Prater, Laura Miles, Angela Corbett, Stephen Koesters. Originally published in the Journal of Medical Internet Research (, 26.06.2018.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.