This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Digital health, which encompasses the use of information and communications technology in support of health, is a key driving force behind the cultural transformation of medicine toward people-centeredness. Thus, eHealth literacy, assisted by innovative digital health solutions, may support better experiences of care.
The purpose of this study is to explore the relationship between eHealth literacy and patient-reported experience measures (PREMs) among users of outpatient care in Hungary.
In early 2019, we conducted a cross-sectional survey on a large representative online sample recruited from the Hungarian general population. eHealth literacy was measured with the eHealth Literacy Scale (eHEALS). PREMs with outpatient care were measured with a set of questions recommended by the Organisation for Economic Co-operation and Development (OECD) for respondents who attended outpatient visit within 12 months preceding the survey. Bivariate relationships were explored via polychoric correlation, the Kruskal–Wallis test, and chi-square test. To capture nonlinear associations, after controlling covariates, we analyzed the relationship between eHEALS quartiles and PREMs using multivariate probit, ordinary least squares, ordered logit, and logistic regression models.
From 1000 survey respondents, 666 individuals (364 females, 54.7%) were included in the study with mean age of 48.9 (SD 17.6) years and mean eHEALS score of 29.3 (SD 4.9). Respondents with higher eHEALS scores were more likely to understand the health care professionals’ (HCPs’) explanations (
We demonstrated the association between eHealth literacy and PREMs. The potential patient-, physician-, and system-related factors explaining the negative experiences among people with highest levels of eHealth literacy warrant further investigation, which may contribute to the development of efficient eHealth literacy interventions. Further research is needed to establish causal relationship between eHealth literacy and patient-reported experiences.
People-centeredness has shaped the cultural transformation of medicine, where we transitioned from a traditional paternalistic model toward a new model of care, grounded in partnerships and putting patients’ values and preferences in the forefront of medical decision making [
Digital health is a key driving force behind the cultural transformation of medicine toward people-centeredness [
Hungary has a tax-funded single-payer health system providing universal health coverage for the population. Most inpatient and specialist ambulatory care services are delivered by the public health system. Primary care—provided by private general practitioners (GPs)—acts as a gatekeeper. Per-capita spending on health care is among the lowest and the share of out-of-pocket contributions including informal payments is among the highest within the European Union. Life expectancy lags behind most European Union countries, mainly driven by lifestyle-related causes. Health inequalities are largely determined by sociodemographic variables [
By making inferences from traditional health literacy studies to the eHealth domain [
This study aims to explore the association between eHealth literacy and OECD’s set of recommend PREMs for users of outpatient care, who were recruited from a large representative online sample from the Hungarian general adult population.
We considered the CHERRIES checklist when reporting this study [
Our sample included those respondents who had a face-to-face appointment with an HCP in the previous 12 months due to their own health problems and answered whether or not the visit had happened at their usual HCP.
eHealth literacy was measured with the Hungarian version of the self-reported eHealth Literacy Scale (eHEALS) [
Respondents’ experiences with ambulatory care were assessed by the set of questions recommended by the OECD’s Health Care Quality Indicators Project [
Following the practice of countries using PREMs for monitoring health system performance, we created composite scores from PREM items [
We recorded respondents’ sociodemographic variables, such as age, gender (female or male), education (primary, secondary, or tertiary), family status (married or not married), employment status (with a paid job or without a paid job including students, pensioners, unemployed, etc), and place of residence (capital, other cities, or village). Age groups were formed according to main Medical Subject Heading (MeSH) categories, adding 18-year olds to the young adult category (young adults: 18–24-year olds; adults: 25–44-year olds; middle aged: 45–64-year-olds; aged > 80: 65+ year-olds) [
Descriptive methods were used when analyzing the sociodemographic characteristics of the sample as well as the PREM items. To test the basic psychometric properties of the PREM scores constructed from multiple items, we assessed their distributional properties, performed exploratory factor analysis (EFA), and calculated internal consistency (Cronbach α) [
We performed multivariate regression analyses to explore the relationship between eHEALS quartiles and PREM items, as well as the composite PREM scores, after controlling for sociodemographic variables, respondents’ health status (MEHM), the setting of the visit (GP, public specialist, or private specialist), and type of HCP (GP, specialist, or other allied health professional). The following models were conducted: (1) logistic regression for binary PREM items or constructed binary variables, (2) ordered logit models for polytomous PREM items, and (3) ordinary least squares (OLS) models for waiting times and composite PREM scores. We tested the joint significance of eHEALS quartiles as a single predictor variable using the Wald test. OLS models were tested for heteroskedasticity via the Breusch–Pagan test and for specification error via the Ramsey regression equation specification error test (RESET). We applied robust regression if heteroskedasticity was detected [
From the 1000 survey respondents, 736 had ambulatory HCP consultation within 12 months, out of which 5 happened over telephone. In 118 cases the respondent did not have a health problem, and 25 respondents could not tell if the visit happened at the regular HCP. After applying all criteria in sequence, 666 individuals were included in the sample (
Sample characteristics.
Characteristics | Sample (N=666), n (%) | Survey (N=1000), n (%) | General adult population [ |
|||||
|
|
|
|
|||||
|
|
|
|
|
||||
|
|
18-24 | 62 (9.3) | 118 (11.8) | 10.6 | |||
|
|
25-44 | 234 (35.1) | 389 (38.9) | 35.7 | |||
|
|
45-64 | 191 (28.7) | 272 (27.2) | 33.1 | |||
|
|
65+ | 179 (26.9) | 221 (22.1) | 20.6 | |||
|
|
|
|
|
||||
|
|
Female | 364 (54.7) | 550 (55.0) | 53.4 | |||
|
|
Male | 302 (45.4) | 450 (45.0) | 46.6 | |||
|
|
|
|
|
||||
|
|
No primary school | —a | — | 0.6 | |||
|
|
Primary | 213 (31.9) | 341 (34.1) | 48.1 | |||
|
|
Secondary | 244 (36.6) | 363 (36.3) | 33.5 | |||
|
|
Tertiary | 209 (31.4) | 296 (29.6) | 17.8 | |||
|
|
|
|
|
||||
|
|
First quintile | 142 (21.3) | 228 (22.8) | 20.0 | |||
|
|
Second quintile | 105 (15.8) | 167 (16.7) | 20.0 | |||
|
|
Third quintile | 57 (8.6) | 81 (8.1) | 20.0 | |||
|
|
Fourth quintile | 86 (12.9) | 118 (11.8) | 20.0 | |||
|
|
Fifth quintile | 186 (27.9) | 254 (25.4) | 20.0 | |||
|
|
Missingb | 90 (13.5) | 152 (15.2) | — | |||
|
|
|
|
|
||||
|
|
Married/domestic partnership | 432 (64.9) | 618 (61.8) | — | |||
|
|
Single/divorced/widow | 234 (35.1) | 382 (38.2) | — | |||
|
|
|
|
|
||||
|
|
Paid job | 319 (47.9) | 500 (50.0) | 48.3 | |||
|
|
Without paid job | 347 (52.1) | 500 (50.0) | 51.7 | |||
|
|
|
|
|
||||
|
|
Budapest | 146 (21.9) | 213 (21.3) | 17.4 | |||
|
|
City | 371 (55.7) | 557 (55.7) | 52.1 | |||
|
|
Village | 149 (22.4) | 230 (23.0) | 30.5 | |||
|
|
|
|
|
||||
|
|
Central Hungary | 236 (35.4) | 348 (34.8) | 30.0 | |||
|
|
Transdanubia | 237 (35.6) | 299 (29.9) | 30.4 | |||
|
|
Great Plain and North | 193 (28.9) | 353 (35.3) | 39.6 | |||
|
|
|
|
|||||
|
|
|
|
|
||||
|
|
Very bad | 3 (0.5) | 5 (0.5) | — | |||
|
|
Bad | 62 (9.3) | 77 (7.7) | — | |||
|
|
Fair | 252 (37.8) | 323 (32.3) | — | |||
|
|
Good | 293 (43.9) | 471 (47.1) | — | |||
|
|
Very good | 56 (8.4) | 124 (12.4) | — | |||
|
|
|
|
|
||||
|
|
No | 200 (30.0) | 390 (39.0) | — | |||
|
|
Yes | 393 (59.0) | 489 (48.9) | — | |||
|
|
Missing | 73 (10.9) | 121 (12.1) | — | |||
|
|
|
|
|
||||
|
|
Not limited at all | 342 (51.4) | 579 (57.9) | — | |||
|
|
Limited but not severely | 254 (38.1) | 313 (31.3) | — | |||
|
|
Severely limited | 46 (6.9) | 56 (5.6) | — | |||
|
|
Missing | 24 (3.6) | 52 (5.2) | — | |||
|
|
|
|
|||||
|
|
|
|
|
||||
|
|
No/not face-to-face/missing | 0 (0.0) | 269 (26.9) | — | |||
|
|
Yes, but not for own health problem | 0 (0.0) | 52 (5.2) | — | |||
|
|
Yes, at regular HCP | 546 (81.9) | 546 (54.6) | — | |||
|
|
Yes, but not at regular HCP | 120 (18.0) | 120 (12.0) | — | |||
|
|
Yes, missing if regular HCP | 0 (0.0) | 13 (1.3) | — |
aNot available.
bMissing: missing responses/do not know/do not want to answer.
cNUTS: Nomenclature of Territorial Units for Statistics.
dMEHM: Minimal European Health Module.
eHCP: health care professional.
Patient responses by PREMa items (N=666).
Patient response | n (%) | ||
|
|
||
|
|
|
|
|
|
GPb | 278 (41.7) |
|
|
Public specialist | 316 (47.4) |
|
|
Private specialist | 72 (10.8) |
|
|
|
|
|
|
GP | 278 (41.7) |
|
|
Specialist | 360 (54.1) |
|
|
Nurse/other HCP | 28 (4.2) |
|
|
|
|
|
|
In the last 30 days | 277 (41.6) |
|
|
Between 1 and 3 months ago | 180 (27.0) |
|
|
Between 3 and 6 months ago | 95 (14.3) |
|
|
Between 6 and 12 months ago | 114 (17.1) |
|
|
||
|
|
|
|
|
|
No | 506 (76.0) |
|
|
Yes | 147 (22.1) |
|
|
Missingd | 13 (2.0) |
|
|
|
|
|
|
No | 534 (80.2) |
|
|
Yes | 120 (18.0) |
|
|
Missing | 12 (1.8) |
|
|
|
|
|
|
No | 559 (83.9) |
|
|
Yes | 99 (14.9) |
|
|
Missing | 8 (1.2) |
|
|
|
|
|
|
No | 508 (76.3) |
|
|
Yes | 148 (22.2) |
|
|
Missing | 10 (1.5) |
|
|
||
|
|
|
|
|
|
No | 487 (73.1) |
|
|
Yes | 179 (26.9) |
|
|
|
|
|
|
No | 564 (84.7) |
|
|
Yes | 102 (15.3) |
|
|
||
|
|
|
|
|
|
Yes, definitely | 427 (64.1) |
|
|
Yes, to some extent | 160 (24.0) |
|
|
No, not really | 57 (8.6) |
|
|
Definitely not | 17 (2.6) |
|
|
Missing | 5 (0.8) |
|
|
|
|
|
|
Yes, definitely | 459 (68.9) |
|
|
Yes, to some extent | 166 (24.9) |
|
|
No, not really | 27 (4.1) |
|
|
Definitely not | 12 (1.8) |
|
|
Missing | 2 (0.3) |
|
|
|
|
|
|
Yes, definitely | 414 (62.2) |
|
|
Yes, to some extent | 164 (24.6) |
|
|
No, not really | 63 (9.5) |
|
|
Definitely not | 15 (2.3) |
|
|
Missing | 10 (1.5) |
|
|
|
|
|
|
Yes, definitely | 338 (50.8) |
|
|
Yes, to some extent | 195 (29.3) |
|
|
No, not really | 77 (11.6) |
|
|
Definitely not | 19 (2.9) |
|
|
Missing | 37 (5.6) |
|
|
|
|
|
|
Poor | 19 (2.9) |
|
|
Fair | 60 (9.0) |
|
|
Good | 186 (27.9) |
|
|
Very good | 205 (30.8) |
|
|
Excellent | 193 (29.0) |
|
|
Missing | 3 (0.5) |
aPREM: OECD-proposed set of questions on Patients’ Experiences with Ambulatory Care.
bGP: general practitioner.
cHCP: health care professional.
dMissing: missing responses/do not know/do not want to answer.
Mean eHEALS score was 29.3 (SD 4.9). eHEALS quartile mean scores were as follows: first quartile 23.5 (range 12-26; 191/666, 28.7%), second quartile 28.2 (range 27-29; 151/666, 22.7%), third quartile 31.2 (range 30-32; 182/666, 27.3%), and fourth quartile 36.0 (range 33-40; 142/666, 21.3%). Mean age of individuals in the fourth eHEALS quartile was 44.5 years (SD 17.1), which was lower than that of individuals in the first (49.9 years [SD 17.4]), second (51.2 [SD 17.5]), and third quartiles (49.3 [SD 17.7;
A majority of respondents (380/631, 60.2%) did not report unmet medical needs in any areas. One unmet need was reported by 18.5% (117/631), 2 unmet needs by 8.9% (56/631), 3 unmet needs by 7.4% (47/631), and 4 unmet needs by 4.9% (31/631) of respondents. The Unmet Medical Needs Score had a single-factor structure with a KMO value of 0.73, suggesting moderately adequate sampling for EFA. The Cronbach α of .73 suggested acceptable internal consistency of this score constructed by adding the PREM items of the Unmet Medical Needs section.
Mean office waiting times were 63.3 (SD 71.0) minutes; 23.0% (152/661) of respondents waited less than 15 minutes, while waiting time was longer than 2 hours for 14.2% (94/661) of the sample. Long office waiting time was a problem for 26.9% (179/666) of all respondents, and for 34.8% (179/514) of those who waited longer than 15 minutes. Mean appointment waiting time was 16.8 (SD 27.8) days. Whereas 37.6% (242/643) of the sample could get an appointment on the same day, 18.2% (117/643) of respondents had to wait longer than 30 days. Long appointment waiting time was a problem for 15.3% (102/666) of all respondents, and for 24.1% (102/424) of those who did not get appointment on the same day. Any waiting problem either at the HCP office or before getting an appointment was reported by 33.5% (223/666) of the sample.
The Problem Score showed strong right skew (mean 2.0, SD 2.5; median 1; kurtosis 4.5; skewness 1.4; Shapiro–Wilk test
The polychoric correlation matrix of PREM items and eHEALS score is shown in
Correlation matrix of PREMa items.b
Variable | Patient experiences | Access to care | |||||||||||
|
|
Waiting times | Unmet medical needs | ||||||||||
|
Time | Understand | Questions | Decisions | Overall quality | oWT | oWP | aWT | aWP | Travel | Visit | Intervention | Medication |
Timec | 1.00 |
|
|
|
|
|
|
|
|
|
|
|
|
Understandd | 0.75 | 1.00 |
|
|
|
|
|
|
|
|
|
|
|
Questionse | 0.77 | 0.77 | 1.00 |
|
|
|
|
|
|
|
|
|
|
Decisionsf | 0.71 | 0.76 | 0.83 | 1.00 |
|
|
|
|
|
|
|
|
|
Overall qualityg | –0.79 | –0.75 | –0.78 | –0.74 | 1.00 |
|
|
|
|
|
|
|
|
oWTh | 0.35 | 0.30 | 0.34 | 0.24 | –0.36 | 1.00 |
|
|
|
|
|
|
|
oWPi | 0.42 | 0.43 | 0.38 | 0.38 | –0.45 | 0.67 | 1.00 |
|
|
|
|
|
|
aWTj | 0.17 | 0.16 | 0.11 | 0.14 | –0.12 | 0.11 | 0.15 | 1.00 |
|
|
|
|
|
aWPk | 0.42 | 0.32 | 0.38 | 0.35 | –0.37 | 0.32 | 0.50 | 0.67 | 1.00 |
|
|
|
|
Travell | 0.20 | 0.21 | 0.21 | 0.24 | –0.16 | 0.17 | 0.33 | 0.05 | 0.38 | 1.00 |
|
|
|
Visitm | 0.36 | 0.30 | 0.36 | 0.27 | –0.32 | 0.14 | 0.40 | 0.10 | 0.36 | 0.61 | 1.00 |
|
|
Interventionn | 0.17 | 0.16 | 0.20 | 0.19 | –0.18 | 0.12 | 0.37 | 0.18 | 0.44 | 0.61 | 0.89 | 1.00 |
|
Medicationo | 0.22 | 0.23 | 0.26 | 0.16 | –0.17 | 0.16 | 0.33 | 0.15 | 0.43 | 0.47 | 0.66 | 0.67 | 1.00 |
eHEALSp | –0.03 | –0.13 | –0.04 | –0.04 | 0.11 | 0.01 | 0.14 | –0.04 | –0.02 | –0.02 | 0.00 | 0.02 | –0.06 |
Last visitq | 0.03 | -0.01 | 0.00 | 0.01 | –0.06 | 0.00 | 0.06 | –0.10 | –0.08 | 0.00 | –0.07 | –0.10 | –0.04 |
aPREM: OECD (Organisation for Economic Co-operation and Development)-proposed set of questions on Patients’ Experiences with Ambulatory Care.
bPairwise tetrachoric correlations for binary item pairs, polychoric correlations for polytomous items, polyserial and biserial correlations between eHEALS scores and polytomous and binary items, respectively.
cDoctor spending enough time with patient in consultation (4-point Likert scale; higher points indicate more problems).
dDoctor providing easy-to-understand explanations (4-point Likert scale; higher points indicate more problems).
eDoctor giving opportunity to ask questions or raise concerns (4-point Likert scale; higher points indicate more problems).
fDoctor involving patient in decisions about care and treatment (4-point Likert scale; higher points indicate more problems).
gOverall quality of last appointment (5-point Likert scale; higher points indicate better experience).
hWaiting time to be seen on the day of consultation (office waiting time [oWT]).
iProblem with waiting to be seen on the day of consultation (office waiting was a problem [oWP]).
jWaiting time to get the appointment (appointment waiting time [aWT]).
kProblem with waiting for appointment: yes (appointment waiting time was a problem [aWP]).
lMissed visit due to travel burden.
mMissed visit due to cost burden.
nMissed intervention due to cost burden.
oMissed medication due to cost burden.
peHEALS: eHealth Literacy Scale.
qTime of last visit: 4 categories; higher points indicate more time elapsed since last visit.
The association between eHEALS quartiles and the problem score was not significant (Kruskal–Wallis test with ties, χ23=4.9,
Problem score by eHEALS (eHealth Literacy Scale) quartiles.
Negative Experiences Score by eHEALS (eHealth Literacy Scale) quartiles.
Perceived easiness of understanding the explanations of the health care professional by eHEALS (eHealth Literacy Scale) quartiles.
Perceived involvement of the respondent by the health care professional in decisions about care and treatment by eHEALS (eHealth Literacy Scale) quartiles.
Mean overall quality of the last visit by eHEALS (eHealth Literacy Scale) quartiles.
Overall quality categories of the last visit by eHEALS (eHealth Literacy Scale) quartiles.
After controlling for sociodemographic variables, respondents’ health status, the setting of the visit, and type of HCP in ordered logit models (
After controlling for covariates, the overall quality also differed between respondents in the first and third eHEALS quartiles (
Ordered logit regression of the patient experience PREMa items.
Variables | Timeb | Understandc | Questionsd | Decisionse | ||||||||
|
|
β | β | β | β | |||||||
|
|
|
|
|
|
|
|
|
||||
|
Second quartile | –0.30 | .28 | –0.29 | .31 | –0.42 | .12 | –0.20 | .47 | |||
|
Third quartile | –0.31 | .24 | –0.98 | <.001 | –0.54 | .04 | –0.30 | .23 | |||
|
Fourth quartile | –0.14 | .61 | –0.51 | .09 | –0.26 | .34 | 0.15 | .57 | |||
|
|
|
|
|
|
|
|
|
||||
|
25-44 years old | –0.87 | .02 | –0.78 | .04 | –0.81 | .03 | –0.73 | .05 | |||
|
45-64 years old | –1.06 | .009 | –1.45 | <.001 | –1.38 | <.001 | –0.88 | .03 | |||
|
65+ years old | –1.38 | .002 | –1.81 | <.001 | –1.58 | <.001 | –1.42 | <.001 | |||
|
|
|
|
|
|
|
|
|
||||
|
Secondary | –0.27 | .31 | 0.10 | .71 | –0.03 | .90 | –0.01 | .98 | |||
|
Tertiary | 0.25 | .37 | 0.22 | .47 | 0.14 | .62 | 0.08 | .78 | |||
|
|
|
|
|
|
|
|
|
||||
|
Male | –0.37 | .09 | 0.12 | .59 | –0.20 | .36 | –0.08 | .68 | |||
|
|
|
|
|
|
|
|
|
||||
|
Second quintile | –0.01 | .97 | –0.31 | .36 | 0.32 | .31 | 0.40 | .19 | |||
|
Third quintile | 0.22 | .57 | –0.17 | .67 | 0.50 | .17 | 0.53 | .14 | |||
|
Fourth quintile | 0.25 | .47 | –0.22 | .54 | 0.24 | .49 | 0.38 | .25 | |||
|
Fifth quintile | 0.08 | .8 | 0.07 | .83 | 0.30 | .32 | 0.49 | .10 | |||
|
|
|
|
|
|
|
|
|
||||
|
Yes | –0.34 | .19 | –0.14 | .60 | –0.13 | .59 | –0.14 | .58 | |||
|
|
|
|
|
|
|
|
|
||||
|
Married/domestic partnership | –0.27 | .21 | –0.22 | .32 | 0.01 | .97 | –0.12 | .55 | |||
|
|
|
|
|
|
|
|
|
||||
|
City | –0.07 | .76 | –0.20 | .44 | 0.10 | .67 | 0.21 | .37 | |||
|
Village | –0.70 | .03 | –0.34 | .31 | –0.29 | .34 | –0.59 | .06 | |||
|
|
|
|
|
|
|
|
|
||||
|
Very bad | –13.30 | .99 | –12.77 | .99 | 0.13 | .92 | 0.03 | .98 | |||
|
Bad | 1.20 | .03 | 1.67 | .02 | 0.72 | .19 | 1.46 | .01 | |||
|
Fair | 0.52 | .26 | 1.80 | .003 | 0.41 | .35 | 0.96 | .04 | |||
|
Good | 0.35 | .41 | 1.35 | .02 | 0.15 | .71 | 0.63 | .14 | |||
|
|
|
|
|
|
|
|
|
||||
|
Limited but not severely | 0.26 | .27 | 0.15 | .54 | 0.11 | .62 | 0.22 | .34 | |||
|
Severely limited | 0.19 | .66 | 0.12 | .80 | 0.33 | .44 | 0.23 | .56 | |||
|
|
|
|
|
|
|
|
|
||||
|
Yes | 0.18 | .49 | 0.20 | .48 | 0.29 | .27 | 0.39 | .12 | |||
|
|
|
|
|
|
|
|
|
||||
|
Public specialist | 1.23 | .009 | 1.23 | .01 | —n | .99 | 0.37 | .46 | |||
|
Private specialist | 0.90 | .11 | 1.00 | .08 | –0.37 | .53 | 0.34 | .54 | |||
|
|
|
|
|
|
|
|
|
||||
|
Specialist | –1.39 | .003 | –1.32 | .005 | –0.41 | .42 | –0.83 | .09 | |||
|
Nurse/other HCP | — | — | — | — | — | — | — | — | |||
|
|
|
|
|
|
|
|
|
||||
|
Yes | –0.29 | .28 | 0.05 | .87 | –0.35 | .18 | –0.27 | .31 | |||
N | 502 |
|
504 |
|
500 |
|
477 |
|
||||
LRq test |
52.7 | .003 | 60.6 | <.001 | 43.7 | .03 | 50.7 | .005 | ||||
GOFr test |
18.5 | .86 | 13.5 | .98 | 21.7 | .71 | 24.9 | .52 |
aPREM: OECD (Organisation for Economic Co-operation and Development)-proposed set of questions on Patients’ Experiences with Ambulatory Care.
bDoctor spending enough time with patient in consultation (4-point Likert scale).
cDoctor providing easy to understand explanations (4-point Likert scale).
dDoctor giving opportunity to ask questions or raise concerns (4-point Likert scale).
eDoctor involving patient in decisions about care and treatment (4-point Likert scale).
fBase: first quartile.
gBase: 18-24 years old.
hBase: primary.
iBase: first quintile.
jBase: capital.
kBase: very good.
lBase: not limited.
mBase: general practitioner.
nNot available.
oBase: general practitioner.
pHCP: health care professional.
qLikelihood ratio; omnibus test for independence, current model versus null model.
rGoodness of fit; ordinal version of the Hosmer–Lemeshow test.
Multivariate regression of PREMa scores.
Model | Overall quality | Log-problem score | Negative experience score | Any negative experience | |||||||
|
Ordered logit | Robustb | Robust | Logistic | |||||||
|
|
β | β | β | β | ||||||
|
|
|
|
|
|
|
|
|
|||
|
Second quartile | 0.24 | .31 | –0.06 | .23 | –0.37 | .08 | –0.25 | .38 | ||
|
Third quartile | 0.55 | .02 | –0.10 | .02 | –0.46 | .02 | –0.16 | .54 | ||
|
Fourth quartile | 0.34 | .16 | –0.02 | .74 | –0.17 | .40 | –0.17 | .56 | ||
|
|
|
|
|
|
|
|
|
|||
|
25–44 years old | 0.56 | .09 | –0.15 | .03 | –0.46 | .08 | –0.64 | .14 | ||
|
45–64 years old | 0.71 | .04 | –0.22 | .002 | –0.83 | .003 | –1.15 | .01 | ||
|
65+ years old | 1.12 | .003 | –0.29 | <.001 | –1.16 | <.001 | –1.60 | .001 | ||
|
|
|
|
|
|
|
|
|
|||
|
Secondary | –0.12 | .60 | —g | .97 | –0.01 | .96 | –0.04 | .89 | ||
|
Tertiary | –0.39 | .10 | 0.04 | .36 | 0.18 | .37 | –0.03 | .92 | ||
|
|
|
|
|
|
|
|
|
|||
|
Male | 0.07 | .69 | –0.03 | .42 | –0.03 | .86 | 0.21 | .32 | ||
|
|
|
|
|
|
|
|
|
|||
|
Second quintile | –0.10 | .70 | 0.04 | .35 | 0.22 | .29 | 0.71 | .03 | ||
|
Third quintile | 0.17 | .59 | 0.05 | .40 | 0.24 | .39 | 0.78 | .04 | ||
|
Fourth quintile | –0.01 | .97 | 0.02 | .69 | 0.20 | .41 | 0.44 | .20 | ||
|
Fifth quintile | 0.06 | .81 | 0.05 | .33 | 0.27 | .21 | 0.61 | .047 | ||
|
|
|
|
|
|
|
|
|
|||
|
Yes | 0.12 | .59 | –0.04 | .31 | –0.08 | .64 | 0.02 | .93 | ||
|
|
|
|
|
|
|
|
|
|||
|
Married/domestic partnership | 0.25 | .17 | –0.03 | .45 | –0.09 | .55 | 0.12 | .59 | ||
|
|
|
|
|
|
|
|
|
|||
|
City | –0.04 | .85 | — | .99 | 0.03 | .85 | 0.10 | .70 | ||
|
Village | 0.26 | .34 | –0.11 | .03 | –0.49 | .02 | –0.66 | .03 | ||
|
|
|
|
|
|
|
|
|
|||
|
Very bad | 0.21 | .88 | –0.05 | .72 | 0.04 | .95 | 0.01 | .99 | ||
|
Bad | –0.97 | .047 | 0.24 | .007 | 1.24 | <.001 | 1.32 | .02 | ||
|
Fair | –1.00 | .01 | 0.15 | .02 | 0.80 | .003 | 0.69 | .11 | ||
|
Good | –0.85 | .02 | 0.10 | .09 | 0.59 | .01 | 0.49 | .21 | ||
|
|
|
|
|
|
|
|
|
|||
|
Limited but not severely | –0.27 | .18 | 0.04 | .28 | 0.17 | .34 | 0.34 | .14 | ||
|
Severely limited | –0.23 | .55 | 0.05 | .48 | 0.20 | .52 | –0.08 | .85 | ||
|
|
|
|
|
|
|
|
|
|||
|
Yes | 0.04 | .85 | 0.04 | .32 | 0.15 | .41 | 0.03 | .91 | ||
|
|
|
|
|
|
|
|
|
|||
|
Public specialist | –0.60 | .16 | 0.18 | .09 | 0.49 | .22 | 0.95 | .12 | ||
|
Private specialist | –0.05 | .91 | 0.13 | .24 | 0.22 | .62 | 0.29 | .66 | ||
|
|
|
|
|
|
|
|
|
|||
|
Specialist | 0.72 | .09 | –0.24 | .02 | –0.75 | .05 | –1.19 | .046 | ||
|
Nurse/other HCP | — | — | — | — | — | — | — | — | ||
|
|
|
|
|
|
|
|
|
|||
|
Yes | 0.03 | .91 | –0.05 | .24 | –0.25 | .22 | –0.36 | .21 | ||
Constant | — | — | 1.87 | <.001 | 1.82 | <.001 | 0.81 | .25 | |||
N | 503 |
|
473 |
|
473 |
|
505 |
|
|||
LRo test |
42.1 | .04 |
|
|
|
|
49.6 | .007 | |||
LR test |
|
|
2.63 | <.001 | 2.27 | <.001 |
|
|
|||
R2 |
|
0.13 |
|
0.13 |
|
|
|||||
GOFp test |
30.5 | .68 |
|
|
|
|
|
|
|||
GOF test |
|
|
|
|
|
|
503.3 | .14 | |||
Ramsey RESETq |
|
|
2.37 | .07 | 0.07 | .98 |
|
|
aPREM: OECD-proposed set of questions on Patients’ Experiences with Ambulatory Care.
bOrdinary least squares (OLS) regression with robust standard errors.
cBase: first quartile.
deHEALS: eHealth Literacy Scale.
eBase: 18-24 years old.
fBase: Primary.
gNot available.
hBase: first quintile.
iBase: Capital.
jBase: Very good.
kBase: Not limited.
lBase: General practitioner.
mBase: General practitioner.
nHCP: health care professional.
oLikelihood ratio; omnibus test for independence, current model versus null model.
pGoodness of fit; Hosmer–Lemeshow test.
qRegression equation specification error test.
The specification of robust linear regression models was acceptable for the log-problem score and the negative experience score (
To our knowledge, this is the first study that explores the relationship between eHealth literacy and PREMs with outpatient care. Our findings show a weak concave relationship between eHEALS scores and PREMs. We observed significant differences between respondents with lowest self-reported eHealth literacy levels (first eHEALS quartile) and the ones with moderately high levels (third eHEALS quartile) in terms of how easy it was to understand the explanations of the HCP, having the opportunity to ask questions, the number of items where respondents experienced problems, and the overall quality of the last visit. Sensitivity analysis using alternative boundaries between eHEALS groups confirmed these findings in multiple alternative scenarios. Although the bivariate association between eHealth literacy and the involvement of respondents in decision making was significant, after controlling for covariates in multiple regression analyses, respondents’ perception of spending enough time in the consultation and involvement in decision making did not show a statistically significant relationship with the eHEALS scores. Besides, our findings show no significant association between eHealth literacy and unmet medical needs and waiting times.
Although our literature search did not reveal papers reporting the association between PREM and eHealth literacy, several studies explored the effect of eHealth literacy (measured with the eHEALS instrument) on aspects of people-centered care such as the patient–physician relationship. A study among Iranian patients with multiple myeloma found a positive relationship between eHealth literacy and shared decision making, where eHealth literacy had a direct positive influence on shared decision making and an indirect positive effect mediated by collaborative patient communication patterns and trust in the health care system [
Our results show a strong negative correlation between the overall quality of the visit and the perceived problems with HCP communication including the involvement of the respondents in decision making. However, the relationship between overall patient-reported experience and eHealth literacy was not linear. The slightly increased probability of negative patient experiences among respondents with highest eHEALS scores is in line with the findings of a large international qualitative study among online health information users, where participants frequently reported reluctance to discuss the online content due to the expected negative reception from their HCPs [
Recognizing the multidimensional determinants of the patient–physician interaction, a recent line of research aimed to establish patient profiles characterized by various skill levels and attitudes, including eHealth literacy [
Among several potential contributing factors, the emergence of negative experiences among patients with greater eHealth literacy levels may partly be explained by the properties of the eHEALS instrument. Showing low correlation with objective measures of eHealth literacy, eHEALS has been described as a tool measuring rather self-efficacy related to eHealth literacy than actual skills [
We also assume that access to high-quality online information including international best-in-class services may raise the expectations of people that may contrast their real-world experiences with the Hungarian health system, which operates at a lower efficiency and expenditure levels compared with other high-income societies [
Although the relationship between eHealth literacy and unmet medical needs or waiting times was not significant in our sample, we found the highest eHEALS scores among respondents with worst self-reported health [
Our study was conducted in the general population without focusing on any particular disease area. Most of the respondents reported on the last ambulatory visit at their usual HCP, therefore our results reflect the general experiences of individuals with outpatient care, regardless of the nature, number, or severity of their health conditions. We applied Hungarian versions of validated instruments that have been used widely in multiple countries, such as the eHEALS or the OECD’s PREM questionnaire. We demonstrated that the composite PREM scores used in our analyses had adequate psychometric properties. However, caution is needed when generalizing our findings beyond the Hungarian setting, due to the differences of health systems, communication culture, or economic status of countries.
Furthermore, a number of limitations of our study have to be highlighted. First, only a small part of the variance of PREM items was explained by our OLS models, suggesting that potentially important determinants of patient experiences remained unexplored in our study. Moreover, eHEALS quartiles were jointly significant predictors only in case of a single PREM item, whereas—despite significant differences between the first and third quartiles—the joint test of eHEALS was not significant in 2 items. Applying refined analytical methods on a larger sample may explain patient-, physician-, and system-related factors that shape patients’ experiences of care and also clarify the relationship between unmet medical needs and eHealth literacy, which yielded mixed results in our study. A further limitation of our study is the wide recall period spanning up to 1 year between the survey and the last patient visit. Recall bias has been reported in connection with patient-experience surveys, raising concerns about the comparability between data collected with different recall periods [
The potential of eHealth to improve the efficiency of health systems has been recognized by policymakers. Low health literacy is a barrier to efficient implementation of eHealth interventions [
As a conclusion, our results suggest that eHealth literacy, a modifiable patient-related factor [
eHealth Literacy Scale (eHEALS).
OECD-proposed Set of Questions on Patient Experiences with Ambulatory Care (PREM).
Logistic regression analysis of unmet medical needs.
Regression analysis of waiting times.
Regression analyses of unmet medical needs and waiting times PREM scores.
Sensitivity analysis.
Analysis of unmet medical needs on extended sample.
This publication was supported by the Higher Education Institutional Excellence Program of the Ministry of Ministry for Innovation and Technology in the framework of the “Financial and Public Services” research project (NKFIH-1163-10/2019) at the Corvinus University of Budapest. The contribution of OBF occurred within a Marie Skłodowska-Curie Innovative Training Network (HealthPros—Healthcare Performance Intelligence Professionals) that has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 765141 (
None declared.