We conducted a survey among a sample of 12,793 patients with diabetes by randomly selecting patients aged 18 to 85 years and asking 1500 patients with a login to the Web portal and 3000 patients without a login to participate. Patients were sent a set of questionnaires and a reminder letter twice if necessary. Patients who did not want to participate were asked to state the reason. The survey was approved by the Medical Research Ethics Committee of the University Medical Center Utrecht (protocol number 11-296/C).

Primary care practices and the regional hospital joined together in an organization called “Diamuraal” to coordinate the care of patients with diabetes in a defined geographical area in the center of the Netherlands. Currently, Diamuraal comprises 62 independent primary care practices and 1 hospital outpatient clinic. All physicians and nurses who participate in the care of patients with type 1 and type 2 diabetes in Diamuraal use the same electronic health record and patients can request a login to access their personal electronic health records. When a patient wants access to the Web portal, he or she needs to sign a registration form which the physician has to cosign. The portal is called “Digitaal Logboek” and was developed by Diamuraal and a private company (Portavita). Patients have access to their diabetes-specific medical records, including information provided by their physician during medical consultation, such as physical examination, laboratory results, problem lists, and treatment goals (Figure 1). Laboratory results are accessible as soon as the laboratories report them in advance of a medical consultation. The Web portal also provides access to general diabetes information and to an overview of all personal diabetes-related examinations and consultations that are needed and/or scheduled. Patients can import and upload the glucose levels measured at home and contact their physician or diabetes nurse through secured electronic messaging. The portal is supplementary; patients who do not request access still receive diabetes care according to the Dutch guidelines. At the start of our survey, 12,793 patients with diabetes were treated in Diamuraal, of whom 9791 (76.53%) never requested a login.

We collected information through a set of questionnaires and by extracting data from the electronic health records. Patient characteristics were obtained from the patients’ electronic health records (age, gender, type of diabetes, duration of diabetes, setting of diabetes treatment, HbA_1c, and total cholesterol) and from the questionnaires (educational level, ethnicity, living status, employment, medication, smoking, drinking, fluency in the Dutch language, and access to computer and Internet). We did not take blood pressure into account because it was not a determinant for portal use [13].

We designed 2 separate questionnaires: 1 for patients with a login and 1 for patients without a login. They were based on characteristics found previously on the use of eHealth in literature [18,19].

The questionnaire for patients with a login contained multiple choice questions about (1) reasons for requesting a login (influence in disease and management of disease, to reread information at home, others thought it would be useful, discontent with current care, other); (2) the way people were informed about the portal’s existence (by a health care provider, a poster, an information pamphlet, through friends or relatives, other); (3) the frequency of portal use (from daily to monthly); (4) the duration of portal use (from less than 15 minutes to more than 1 hour); and (5) the person who added the information to the portal (the user/self, family, friends, or others).

The questionnaire for patients without a login contained questions about their awareness of the patient Web portal and, if applicable, how people were informed about its existence (see above) as well as the reasons for nonuse (all yes/no questions). There was room for free text as well.

Both questionnaires contained questions about the use of the Internet for other purposes than the Web portal, with regard to frequency and duration and the use of the Internet for searching information about diabetes (from never to monthly). The 9 questions about the perceived usefulness of specific portal components were answered on a 5-point Likert scale ranging from very important to unimportant.

The final question regarded possible improvements to the portal. All recommendations were scored on a 5-point Likert scale ranging from very important to unimportant. There was room for free text.

Questions about specific portal components and the question about recommendations regarding possible improvements were, in case of nonusers, addressed as how nonusers expected the usefulness of that particular component to be.

In addition to these specifically designed questionnaires, the set of questionnaires contained additional validated questionnaires, including the Problem Areas in Diabetes (PAID) questionnaire to measure diabetes-specific distress [20,21], the Diabetes Treatment Satisfaction Questionnaire (DTSQ) to measure satisfaction with diabetes treatment [22], the Diabetes Management Self-Efficacy Scale (DMSES) to measure self-efficacy [23], and the Brief Diabetes Knowledge Test (BDKT) to measure diabetes knowledge [24,25].

Reason for regular use or nonuse, the answers about content and usefulness of the portal, and about the recommendations were expressed as percentages. The answer categories useful and very useful were combined. The question about reason for nonuse was misread by some patients. We asked for the main reason (1 reason) why a patient did not request a login and provided multiple answers. A total of 59 patients gave more than 1 reason. We used all these answers in the analysis.

We compared patients who requested a login and used it 2 or more times (regular users) and patients who did not request a login at all (nonusers). We decided to perform the analysis only on the regular users instead of all patients with a login because we wanted to compare the patients without a login to a group of patients with actual experience with the patient portal. Based on previous research, we considered the group of nonusers too different from patients who had requested a login but never logged in or logged in only once, the so-called “early quitters.” Indeed, early quitters differed from nonusers: they were younger (mean 61.9, SD 12.7 years vs mean 64.7, SD 10.0 years; P=.001), more often male (63.9%, 140/219 vs 56.5%, 428/758; P=.049), and had a higher educational level (39.4%, 84/213 vs 28.2%, 519/723; P=.002).

Age and gender of nonparticipants (patients who responded but declared that they did not want to participate) and nonresponders (patients who did not respond to the invitation to fill out the questionnaire) of the study are described elsewhere [13].

We used chi-square tests for all categorical variables and unpaired t tests for all normally distributed continuous variables and Mann-Whitney tests for nonnormally distributed continuous variables. Categorical variables were expressed as numbers with percentages and continuous variables as means with standard deviation (SD) or with median and interquartile range (IQR) when not normally distributed. Multivariable logistic regression analysis, using the enter method, was used to determine the adjusted association between patient characteristics and not requesting a login. We used a P value of <0.2 in the univariable analysis to select variables for further multivariable analysis. These determinants were expressed as odds ratios (ORs) with corresponding 95% confidence intervals (95% CI). Data was analyzed using SPSS for Windows version 20 (SPSS Inc, Chicago, IL, USA).

From the 4500 questionnaires, 101 were not answered because 33 patients died and 68 had incorrect contact information. From the remaining patients, 2931 (66.63%) responded; 1541 of these 2931 patients (52.59%) declared that they did not wish to participate. In total, 1390 (31.60%) patients were eligible for analysis (“participants”) because they returned a completed questionnaire and signed a consent form. Their mean age was 63.9 (SD 12.2) years (nonparticipants: mean 64.5, SD 13.8 years; P=.11) and 826 of 1390 (59.42%) were male (nonparticipant group: 1539/3009, 51.15% male patients; P<.001). Of the 1390 participants, 632 (45.47%) had a login and 758 (54.53%) did not (“nonusers”).

The login frequency of the patients with a login was a mean 10.4 (SD 23.0) times and 413 of 632 (65.3%) patients accessed the patient Web portal 2 or more times (“regular users”). The latter category differed in many characteristics from nonusers (Table 1). Of the 94 patients with type 1 diabetes, only 13 (14%) were nonusers, whereas 745 of 1077 (69.17%) patients with type 2 diabetes were nonusers (P<.001). There was also a difference in treatment setting: 666 of 846 (78.8%) patients treated by a general practitioner were nonusers, whereas only 92 of 325 (28.3%) patients treated by an internist were nonusers (P<.001).

The use of the Internet differed between both groups: 321 (77.9%) of the 413 regular users used the Internet daily versus 346 (67.6%) of the 512 nonusers with Internet access (P<.001). When using the Internet, 184 (44.6%) of the 413 regular Web portal users were online for more than an hour per day compared with only 140 (27.3%) of the 512 nonusers (P<.001). Furthermore, 206 (51.1%) of the regular users declared that they used the Internet for searching for information about their disease compared with only 126 (25.4%) of the nonusers (P<.001).

Of the 413 regular users, 328 (79.4%) patients declared that they were the main user of the Web portal themselves and 79 (19.1%) declared that someone else had access to the Web portal and usually accessed the portal. Of the 758 nonusers, 162 (21.4%) patients stated that they would consider using the Web portal if someone would help them, 262 (34.6%) did not know if they would use the portal if someone would help, and 293 (38.7%) would not consider using the portal even if someone would help.

Multivariable analysis showed that increasing age and smoking were associated with not using the Web portal. On the contrary, a higher educational level, treatment by an internist, using insulin, polypharmacy, better diabetes knowledge, and more hyperglycemic episodes were less likely to be associated with not using the Web portal (Table 2).

The main reason for not requesting a login was that 549 of 758 (72.4%) patients were not aware of the portal’s existence. Another 216 of 758 (28.5%) stated that the main reason for not requesting a login was that they preferred to leave the disease management to the physician (Table 3).

The reasons for requesting a login among the regular users were to reread information of the consultation at home (312/413, 75.5%), the feeling that the portal use would give them influence on their disease and treatment (132/413, 32.0%), the fact that the physician or someone else thought the Web portal could be useful for them (74/413, 17.9%), dissatisfaction with the current care (2/413, 0.5%), and other reasons (27/413, 6.5%).

Of the 209 patients without a login who stated they were aware of the existence of the portal, 102 (48.8%) knew about the portal because their health care provider told them. In comparison, 392 (94.9%) of the 413 regular users were informed about the portal by their health care provider (P<.001). Other sources of information about using the Web portal were posters in the clinic waiting area (nonusers: 10/209, 4.8%; regular users: 7/413, 0.7%; P<.001), a pamphlet (nonusers: 4/209, 1.9%; regular users: 15/413, 3.6%; P=.24), friends or relatives who used the portal themselves (nonusers: 20/209, 9.6%; regular users: 4/413, 1.0%; P<.001), and other reasons (nonusers: 29/209, 13.9%; users: 9/413, 2.2%; P<.001).

Regular users perceived the usefulness of specific portal content in a different way compared to nonusers (Figure 2). Users perceived access to the laboratory values with treatment targets, the possibility of rereading clinic consultations, and having a summary of all controls as the most useful features of the portal. We asked the nonusers if they could speculate on the possible usefulness of portal features for their own disease management. They suggested a summary of upcoming consultations and a summary of their medication to be the most useful parts of a Web portal.

Regular users and nonusers appreciated additional functionalities that could improve the Web portal differentially (Figure 3). Regular users wanted to be able to add their injected insulin units to the glucose diary and to use the portal for supportive care, such as scheduling an appointment and receiving reminders about upcoming consultations. Nonusers felt that a diabetes Web portal could benefit mostly from more information about medication and side effects and they wanted to use the portal for medication refills. Overall, regular users scored more possible features as useful or very useful than nonusers did except for information provided in different languages.