This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Patient portals have drawn much attention, as they are considered an important tool for health providers in facilitating patient engagement. However, little is known about whether the intensive use of patient portals contributes to improved management of patients’ health in terms of their confidence in acquiring health information and exercising self-care. There is a lack of randomized trials with these outcomes measured both pre- and postadoption of patient portals.
The aim of this study was to examine the causal relationship between the usage of patient portals and patients’ self-efficacy toward obtaining health information and performing self-care.
This study was a secondary data analysis that used data from a US national survey, the National Cancer Institute’s Health Information National Trends Survey 5 Cycle 1. Patient portal usage frequency was used to define the treatment. Survey items measuring self-efficacy on a Likert-type scale were selected as the main outcomes, including patients’ confidence in obtaining health information and performing self-care. To establish causality using survey data, we adopted the instrumental variables method. To determine the direction of the causal relationship in the presence of high-dimensional confounders, we further proposed a novel testing framework that employs conditional independence tests in a directed acyclic graph. The average causal effect was measured using the two-stage least squares regression method.
We showed that frequently using patient portals improves patients’ confidence in obtaining health information. The estimand of the weighted average causal effect was 0.14 (95% CI 0.06-0.23;
The results support the use of patient portals and encourage better support and education to patients. The proposed statistical method can be used to exploit the potential of national survey data for causal inference studies.
Given the growing evidence showing that patient engagement improves health outcomes and reduces health care costs, patient portals have drawn much attention. A patient portal is a secure online health platform linked to a patient’s personal medical record that is 24/7 accessible from any location with an internet connection. Patient portals are considered an important tool for health providers in facilitating patient engagement [
Nonetheless, little is known about whether patient portal usage contributes to the improved management of patients’ health in terms of health literacy, communication, confidence in acquiring health information, and the self-monitoring and self-care of health. Interviews have indicated that patient’s perception of access to online records are associated with a greater focus on their health and more proactive involvement in self-care [
Despite the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act and the recommendations by the US Institute of Medicine [
This study was a secondary data analysis that used data from the National Cancer Institute’s HINTS 5 Cycle 1. HINTS 5 Cycle 1 is a cross-sectional survey of a nationally representative sample of US adults used to assess the impact of the health information environment. The survey was conducted from January 2017 through May 2017 using a self-administered mail questionnaire. Out of 10,265 surveys sent out, data were collected from 3285 (32% response rate) respondents [
Patients’ self-efficacy in obtaining health information and performing self-care were considered as the outcomes. Survey items measuring patient self-efficacy were selected based on the Institute of Medicine’s recommendations for promotion of patient portals to increase quality of care and reduce medical errors [
These measures were captured on a 5-point Likert-type scale where a higher score indicated greater confidence. Variables on patients’ age, gender, race, ethnicity, marital status, education, employment status, household income, and insurance status were considered to be confounders in the study. Information on patients’ portal activities was elicited by the following questionnaire item: “How many times did you access your online medical record in the last 12 months? (0/1 to 2 times/3 to 5 times/6 to 9 times/10 or more times).”
Different usage frequencies were considered to be different levels of treatment. For the instrumental variable (IV), we used the following questionnaire item: “Have any of your health care providers including doctors, nurses, or office staff ever encouraged you to use an online medical record? (yes/no).”
Missing values were sparse and were handled in several ways depending on the variable type: samples missing the outcomes were discarded; for the IVs, missing responses were replaced by “no encouragement”; and for the confounders, missing responses were imputed by the Multivariate Imputation by Chained Equations (MICE) [
We aimed to test the hypothesis that the exposure to patient portals, or intensively using a patient portal, will improve patients’ self-efficacy outcomes. However, since we only used a one-time outcome measurement for each individual, we could not construct the before-after treatment contrast to directly measure the treatment effect. Further, without a randomized experimental design, the causality can be obscured by confounders. To address these issues, we adopted the IV method for causal inference in observational studies [
We identified the following item as the binary variable and used it as the IV: “Have any of your health care providers including doctors, nurses, or office staff ever encouraged you to use an online medical record?”
Encouragement plays an influential role on patients’ use of portals, as care providers’ endorsement is an important factor in the adoption of these tools [
To further determine whether patient portal usage causes an improvement in self-efficacy, or vice versa, we proposed a testing framework that could both address the confounding issue and determine the direction of the causal relationship. Our testing framework was based on causal directed acyclic graphs (DAGs), which are used as a graphical tool to visually represent and understand the concepts of exposure, outcome, causation, and confounding [
The proposed DAG-based testing framework aimed to qualitatively evaluate the causality. To quantify the treatment effect, two-stage least squares (TSLS) regression models were built. As the treatment (portal usage) has multiple levels (eg, 1-2 times and 3-5 times annually are different levels), the traditional average treatment effect is not identifiable. However, Angrist and Imbens [
After removing the samples with too many missing data, we identified 3198 participants among the 3285 survey respondents. Among these 3198 participants, 1003 (31%) were self-reported patient portal users. For demographic and socioeconomic variables, the user group and the nonuser group had different characteristics (see Table S1 in
Next, we characterized the users’ portal usage behavior. Of the 1003 portal users, 49% (496) reported using portals 1-2 times in the past 12 months, 31% (313) reported using them 3-5 times, 10% (104) reported using them 6-9 times, and 9% (90) reported using them more than 9 times.
Of the 3198 participants, 1375 (43%) were encouraged to use patient portals and 1823 (57%) were not. Among the 1375 respondents who were encouraged, 549 (40%), 383 (28%), 267 (19%), and 176 (13%) individuals never used a portal, used a portal 1-2 times, 3-5 times, and more than 5 times, respectively. In contrast, there were 1646 (90%), 113 (6%), 46 (3%), and 18 (1%) participants in the nonencouraged group, respectively. It was evident that the IV and the treatment were significantly associated, which was verified by a chi-square test (
There were more patients with positive responses for the self-efficacy outcomes in the patient population who were recommended to use portals. The distributions of each outcome variable conditioning on the value of the IV are displayed in
Chi-square test results for the association between encouragement to use patient portals and self-efficacy outcomes.
Outcomes | IVa: encouragement to use patient portals | |||
|
Yes, n (%) | No, n (%) |
|
|
|
1353 (100) | 1758 (100) | <.001 | |
|
Not confident at all | 21 (1.5) | 50 (2.8) |
|
|
A little confident | 56 (4.1) | 108 (6.1) |
|
|
Somewhat confident | 369 (27.3) | 600 (34.1) |
|
|
Very confident | 566 (41.8) | 607 (34.5) |
|
|
Completely confident | 341 (25.2) | 393 (22.4) |
|
|
1364 (100) | 1801 (100) | .001 | |
|
Not confident at all | 15 (1.1) | 31 (1.7) |
|
|
A little confident | 31 (2.3) | 79 (4.4) |
|
|
Somewhat confident | 320 (23.5) | 469 (26.0) |
|
|
Very confident | 652 (47.8) | 815 (45.3) |
|
|
Completely confident | 346 (25.4) | 407 (22.6) |
|
aIV: instrumental variable.
Following the testing procedure described in
Results of conditional independence tests.
Outcomes | ||
Test A: Z ⊥ Y ∣ |
Test B: Z ⊥ Y ∣ |
|
ConfidentGetHealthInfo | .28 | .02 |
OwnAbilityTakeCareHealth | .42 | .23 |
For ConfidentGetHealthInfo, we could not reject the hypothesis for Test A (overall
The causal relationship among encouragement, portal usage, and confidence in seeking health information with the confounders. IV: instrumental variable.
A TSLS regression model was built to quantify the treatment effect of portal usage on self-efficacy toward acquiring health information. The estimand of the weighted average causal effect was 0.14 (95% CI 0.06-0.23;
Existing work on survey analysis mainly focuses on the strength of the association between questionnaire items and targeted outcomes [
Although we should be cautious in interpreting the relationship between using patient portals and self-efficacy as causal, we did observe a co-occurrence of better outcomes and increased patient portal usage intensity. Despite the benefits of patient portals being well documented [
Our analysis shows that being encouraged to use patient portals positively affects the intensity of portal usage, which in turn influences people’s confidence in acquiring health information. Other studies have reported that actively using patient portals, such as sending messages to physicians and viewing prescription and lab results, is positively associated with high-quality physician-patient relationships and patients’ confidence to understand health information. It is evident that a trusting physician-patient relationship helps promote healing and remove barriers to obtaining medical information [
The study was conducted using HINTS 5 Cycle 1 data, which were collected in 2017. The study can be improved by combining multiple data sets, including ones published recently [
The establishment of the causal relationship between portal usage and patients’ self-efficacy demands that encouragement to use portals is not based on patients’ self-efficacy. We have observed that many health care organizations have integrated the facilitation of portal enrollment into their new patients’ registration protocol [
Although the confounders between the treatment and the outcome can be completely unmeasured, we still require the common confounders to be fully observable. The identification of common confounders largely relies on domain knowledge. It is worth noting that the choice does not have to be unique. In our casual diagram, adding variables to the common confounder set still results in a valid choice. However, chi-square tests can be sensitive to the choice of confounders when the sample size is not sufficiently large [
Since the Affordable Care Act mandated portal usage, enthusiasm for portals has declined; however, this study found that using patient portals improves patients’ confidence in obtaining advice or information about health or medical topics. Our findings thus attest to the benefit of patient portals and to providing better support and education to patients. In addition, our proposed statistical method exploits the potential of using national survey data such as the HINTS program to examine causal effects to obtain new insights. Theoretically, the treatment effect can be heterogeneous based on different patient characteristics. There is thus a need to develop a testing framework that can identify the disparity in causal effects. For justifying the clinical insights identified in this study, we cannot solely rely on patients’ self-reported outcomes, but should also survey physicians on their perception of patients’ self-efficacy. Furthermore, how physicians make encouragement decisions should also be investigated, and a randomized controlled study with pre- and posttreatment outcomes being clearly documented is necessary to fully understand the treatment effect on self-efficacy outcomes.
Table S1.
Methods.
directed acyclic graph
Health Information National Trends Survey
Health Information Technology for Economic and Clinical Health
instrumental variable
Multivariate Imputation by Chained Equations
two-stage least squares
None declared.