Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 17.12.14 in Vol 16, No 12 (2014): December

This paper is in the following e-collection/theme issue:

    Original Paper

    Which Frail Older Patients Use Online Health Communities and Why? A Mixed Methods Process Evaluation of Use of the Health and Welfare Portal

    1Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, Netherlands

    2Jeroen Bosch Hospital, Department of Internal Medicine, 's-Hertogenbosch, Netherlands

    Corresponding Author:

    Peter Makai, MSc, PhD

    Radboud University Medical Center

    Radboud Institute for Health Sciences

    Renier Postlaan 4

    Nijmegen, 6500 HB


    Phone: 31 24 36 19807

    Fax:31 024 361 74 08



    Background: Frail older people often receive fragmented care from multiple providers. According to the literature, there is an urgent need for coordination of care. Online and eHealth tools are increasingly used to improve coordination. However, there are significant barriers to their implementation in frail older people.

    Objective: Our aim was to (1) evaluate differences in use of a personal online health community (POHC) for frail older people in relation to personal characteristics, and (2) explore barriers and facilitators for use as experienced by older people and their informal caregivers, using the case of the Health and Welfare Information Portal (ZWIP).

    Methods: This is a mixed methods study. For the quantitative analysis, we used POHC usage information (2 years follow-up) and baseline characteristics of frail older people. For the qualitative analysis, we used semistructured interviews with older people and their informal caregivers. Participants were recruited from 11 family practices in the east of the Netherlands and frail older people over 70 years. The ZWIP intervention is a personal online health community for frail older people, their informal caregivers, and their providers. ZWIP was developed at the Geriatrics Department of Radboud University Medical Center. We collected data on POHC use for 2 years as well as relevant patient characteristics. Interview topics were description of use, reasons for use and non-use, and user profiles.

    Results: Of 622 frail patients in the intervention group, 290 were connected to ZWIP; 79 used ZWIP regularly (at least monthly). Main predictors for use were having an informal caregiver, having problems with activities of daily living, and having a large number of providers. Family practice level predictors were being located in a village, and whether the family practitioners had previously used electronic consultation and cared for a large percentage of frail older people. From 23 interviews, main reasons for use were perceiving ZWIP to be a good, quick, and easy way of communicating with providers and the presence of active health problems. Important reasons for non-use were lack of computer skills and preferring traditional means of consultation.

    Conclusions: Only 27.2% (79/290) of frail older enrolled in the POHC intervention used the POHC frequently. For implementation of personal online health communities, older people with active health problems and a sizable number of health care providers should be targeted, and the informal caregiver, if present, should be involved in the implementation process.

    Trial Registration: International Standard Randomized Controlled Trial Number (ISRCTN): 11165483; (Archived by WebCite at

    J Med Internet Res 2014;16(12):e278




    Potential Benefits of eHealth for Frail Older People

    Frail older people have a large number of health deficits and receive fragmented care, often resulting in nursing home admissions and hospitalization [1]. Frail older people also have multiple health care providers. The collaboration between these providers is not always optimal, and they are often unaware of the fact that they care for the same patient. This reveals insufficient coordination in the care for frail older people [2]. There are several barriers to providing coordinated care to frail older people in a cross-institutional setting: (1) the physical barriers of working in different institutions located in different geographical locations, (2) not having a timely overview of all professionals involved in the care of the patient, (3) information that colleagues have on the patient, (4) how they treat the patient, and (5) that multidisciplinary consultation can take place only if all professionals are available at the same time. The required coordination of care for frail older people may be facilitated by the implementation of eHealth [3,4].

    Personal online health communities (POHCs) seem particularly suited to improving the coordination of care for frail older people with multiple providers. Such communities allow a patient, and/or professionals caring for a specific patient, to interact in online networks. In general, online health communities are online platforms that unite providers and/or patients with a shared goal or similar interest [5]; in the case of POHCs, that is the care for an individual. POHCs enable communication between people who would not have met each other otherwise [5]. They also provide an alternative to face-to-face consultations. POHCs facilitate communication by organizing a network of providers around a patient [5]. They are a relatively new development in eHealth [6], and so information on the determinants of usage is limited. We developed the Health and Welfare Information Portal (in Dutch: Zorg- en WelzijnsInfoPortaal or ZWIP), a POHC aimed at reducing fragmentation in care in frail older people. ZWIP targets individual patients, their informal caregivers, and their health care providers, to learn about the feasibility and effects of POHCs in the care for frail older people. In an earlier study, we showed that ZWIP had not yet yielded any benefits [7], but there were indications that usage was heterogeneous across users and, in general, limited. Usage is therefore the focus of this study.

    Determinants of Using Personal Online Health Communities in Older People

    Despite the potential benefits of POHCs, implementing eHealth is difficult, especially in frail elderly populations [3]. Implementation difficulties in this group arise because frail older people have decreased physical and cognitive function leading to a loss of autonomy. Older people in general have a lower level of computer literacy than younger populations, which has to be considered when developing and implementing eHealth interventions for this group [8]. Furthermore, older patients may not be able to use complex and multifunctional eHealth interventions, which also leads to low levels of usage.

    At present, little is known about facilitators of eHealth usage in general and POHCs in particular in frail older people and how the barriers presented can be overcome [3]. Previous studies focused on implementation barriers during the introduction of eHealth interventions in other groups, such as health care providers [9-14] and in younger patients with chronic illnesses [15-17]. Knowledge about such determinants of usage in frail older people would allow identification of older people who are likely to use and possibly benefit from eHealth interventions.


    The aim of this study was to investigate the main determinants of ZWIP usage in frail older people. First, we describe ZWIP use and its users’ characteristics. Second, we investigate the predictors (barriers and facilitators) of ZWIP usage in frail older people to identify successful users.



    ZWIP is a personal online health community for multidisciplinary communication and information exchange for frail older people and their informal caregivers. The development of the intervention has been described elsewhere [2]. Briefly, ZWIP is a secure digital environment, aiming to improve collaboration within the care network around frail older people. Within the ZWIP network, frail older people (or on their request, their informal caregiver) take the lead. Patients and their informal caregivers give permission to providers to join their network. For all members this creates an overview of the providers involved in their care. ZWIP contains a message system similar to email, where patients can exchange messages with these providers. Most messages are visible to all members of a patient’s network, with the exception of private messages. Therefore, all relevant professionals and informal caregivers are kept up to date on care-related developments. Based on the family practice’s information system and frailty identification through Easycare-TOS (see below), ZWIP contains current medical and social care data on frail older people, which is shared within a patient’s network. ZWIP also gives the opportunity to register individual patient’s care-related goals and action plans. Goals and action plans are determined by the patients in consultation with the professionals. Patients may also receive tailored health information via ZWIP. (See Multimedia Appendix 1 for a video of a patient using ZWIP.)

    Centers, Participants, and Recruitment

    ZWIP was implemented in 11 family practices in the east of the Netherlands. The participating practices started by screening all of their patients aged 70 years and older in alphabetical order using the Easycare-TOS (Two step Older persons Screening) instrument. Easycare-TOS is a validated, two-step measure to identify frail older people in the community as a target population for integrated geriatric care [18]. Easycare-TOS is based on the EASYcare-system, a focused geriatric assessment by a primary care nurse [19,20]. Initially, the family practitioner completed a questionnaire in Step 1 and judged patients as frail, frailty uncertain, or not frail based on readily available information. If patients were assessed as being frail or family practitioners were uncertain, a second assessment step (Step 2) was carried out by specialized nurses to confirm or exclude frailty. All 622 patients who were considered frail based on the screening were invited to participate in ZWIP, of which 290 (46.6%) consented.


    In complex interventions, which are usually difficult to describe and replicate, qualitative research can help clarify the processes of implementation and change. Therefore, we evaluated ZWIP usage with a mixed methods study. The local ethics committee (Committee on Research Involving Human Subjects Region Arnhem-Nijmegen) stated that no formal approval was required due to the non-invasive and non-experimental nature of the study [21,22].

    Quantitative Data Collection

    Quantitative data were collected in the intervention arm of an effectiveness trial [7], and measurements were performed by trained nurses in the patients’ homes using a face-to-face questionnaire. We used baseline data for demographics, physical, mental and social functioning, as well as coordination of care and care use. We included patient characteristics observed by the family practitioner or obtained during the Easycare-TOS assessment. Individual baseline characteristics used were the Barthel index of Activities of Daily Living (ADL) and the Lawton scale of Instrumental Activities of Daily Living (IADL), the Short Form-36 Health Survey (SF-36) (social activity limitations and mental health) [23-26], as well as education, age, and sex. Family practice level characteristics were practice location, family practitioner having used electronic consultation previously, and percentage of frail older people in the practice. Furthermore, we used data from the ZWIP application’s logging files. In this model, ZWIP usage was operationalized as monthly total page views by the patients or their informal caregivers. ZWIP use was followed up to 2 years. We operationalized ZWIP usage as a combined measure of total page views by the patients or their informal caregivers. To capture all forms of usage, we counted passive usage (simply logging in without interaction with the providers, viewing health promotion) and active usage by the older people or their informal caregivers (actual communication with providers, adapting goals).

    Quantitative Data Analysis

    Baseline characteristics of ZWIP participants and non-participants were compared using t tests and chi-square tests. We classified different groups of users according to usage of ZWIP, using hierarchical cluster analysis [27], creating groups from observations based on similarities of their features. ZWIP users were clustered according to the degree of use of the various ZWIP features (communication, goals, health promotion, etc) [27]. Clusters were created using the Ward method [28], and the number of separate groups were determined using the pseudo t2 and the pseudo F2 [27,29].

    Furthermore, we investigated the relationship between usage and baseline characteristics at the patient and family practice levels. We used a hierarchical linear growth model [30] to account for potential variation in use across time (months since connection to ZWIP) within a patient, between patients with various characteristics, and between family practices. All analyses were performed using SAS 9.2.

    Qualitative Data Collection

    Participants were patients and their informal caregivers who participated in ZWIP and were selected using purposive sampling [31]. To include a wide variety of users and levels of use during sampling, we allowed for the following patient characteristics: sex, age, family practice, whether a patient or an informal caregiver managed the patient’s ZWIP account, network size, and how much they used ZWIP (ZWIP usage). If any patient declined to participate, a patient with comparable characteristics was chosen instead. The final sample size was dictated by data saturation [32].

    Three research assistants, who were not otherwise involved in the evaluation of ZWIP, approached the selected patients or their informal caregivers and conducted individual semistructured interviews at the patients’ homes. The interviewers used a topic list based on relevant themes derived from the literature [33], which evolved when relevant new topics came up. Initial topics were description of use, reasons for use/non-use, and user profiles. New topics were discussed in the research group and pursued if relevant to the aim of this study. All interviews were audiotaped and transcribed verbatim. Data collection continued until saturation was reached (eg, when no new themes emerged from the interviews).

    Qualitative Data Analysis

    To analyze the transcripts, 3 researchers (PM, SR, and MP) used directed content analysis [34] and deconstructed the interviews independently, using the principles of iterative comparison. In discussion meetings, PM, SR, and MP reached agreement on codes, combined codes to generate and adjust categories (axial coding), merged categories to compose and refine themes (selective coding), and discussed whether saturation had been reached. We used Atlas-ti 5.2 software to help in data coding and retrieval.

    Integration of Quantitative and Qualitative Data

    Quantitative data were used to identify qualitative respondents. We based the user profiles on the clusters identified during cluster analysis, thus making the link between qualitative and quantitative data. Qualitative and quantitative data on facilitators for use were presented separately and integrated by triangulation [35] in the discussion. Similar weight was given to the two types of analyses.



    A total of 290 vulnerable older patients (46.6% of all 622 eligible patients) completed baseline measurement and participated in ZWIP. The other 332 refused participation. ZWIP participants were similar to non-users in terms of demographic criteria, functional measures such as ADL and mental health measures, number of illnesses, and the types of illnesses present with the exception of dementia. Furthermore, they were similar in terms of a number of process indicators such as patient experience with care measures, and family practitioner reported process measures, with one exception: patients using ZWIP were more likely to have more providers involved in their care (Table 1).

    In total, 23 patients and informal caregivers participated in the interviews. Patient age varied between 74 and 90 years, and they came from a wide range of practices (see Multimedia Appendix 2). Additionally, as a result of purposive sampling, interviewed patients varied substantially in terms of messages sent and received and if they had an informal caregiver and the type of informal caregivers (spouse or child).


    The most frequently used ZWIP function was messaging with an average of 12 messages per patient during the 2-year follow-up period (Table 2). Additionally, 47.9% (139/290) of patients used the goal-setting function and 13.1% (38/290) of older people modified or evaluated the goals during ZWIP usage. Furthermore, 33.1% (96/290) of patients had defined concrete care-related activities to reach their goals; in the case of 9.0% (26/290) of patients, these activities were modified or actively evaluated in ZWIP during the period of ZWIP usage.

    Hierarchical cluster analysis identified four main groups of users: Non-active users (using ZWIP only a few times within 2 years), Regular users (once a month), Active users (once a week), and a small group of Very active users (daily users). From quantitative demographic data and qualitative data, average user profiles per cluster were constructed (Textbox 1). As can be seen in Figure 1, after an enthusiastic start the number of page views decreased to remain relatively stable in all groups a few months after the start, except for the very active users (Figure 1).

    In addition, qualitative data further characterized the variations in use seen in the quantitative data trends. Qualitative data confirmed that patients and informal caregivers made highly variable use of ZWIP: “I check ZWIP every day” [Respondent 21, Patient], “I probably checked ZWIP 2 or 3 times” [Respondent 6, Informal caregiver], and “At the start I checked ZWIP once or twice a week. Later, it decreased” [Respondent 16, Patient].

    The number of providers that patients had contact with was also highly variable. Whereas some patients never logged in, only added informal caregivers to their network, or waited for others to communicate, other patients used ZWIP intensively to communicate in their network: “Only the family practitioner and district nurse, with the other health care providers I did not have any contact” [Respondent 14, Informal caregiver].

    Other patients mentioned that providers did not respond to the invitations for their networks and that they got little response to their messages from providers. Interviewed ZWIP users and informal caregivers used only the communication platform: “I only used the communication tool” [Respondent 10, Informal caregiver].

    Figure 1. Average page views per month of usage for the 4 identified usage clusters.
    View this figure
    Table 1. Comparison of frail older patients who used and did not use ZWIP.
    View this table
    Table 2. Usage of the different functionalities of ZWIP, as used in the cluster analysis for the four different groups of users.
    View this table

    Textbox 1. Example user profiles per group.
    View this box

    Perceptions of ZWIP Usage

    Perceptions of ZWIP usage were rather positive, illustrated by the quotes below. Some patients and informal caregivers perceived that communication was quicker than by telephone and that ZWIP was a good way to co-decide on their own health and welfare issues with the provider: “It is fast, multidisciplinary and you are in control of your own care. Very pleasant!” [Respondent 22, Patient] and “You send a message, and when it is convenient for them they answer. I think it is easier for them, also for the family practitioner” [Respondent 20, Informal caregiver].

    Exchange of information and access to providers was perceived to be better using ZWIP: “I think it (ZWIP) causes unnecessary appointments to be avoided and at the same time it provides a safe connection to the family practitioner” [Respondent 17, Patient].

    Informal caregivers said the ZWIP gave them a complete overview of the patient’s situation: “You can paint a more complete picture than on the phone.” [Respondent 10, Informal caregiver] and “My parents needed more care. It seemed like a good idea to get a good overview and simplify communication” [Respondent 10, Informal caregiver].

    Other participants were disappointed by the frequency of communication through ZWIP: “Well, we expected that there would be more information exchange...” [Respondent 20, Informal caregiver].

    Predictors for Use: Barriers and Facilitators

    In quantitative user analysis, several determinants of use were identified (Table 3). When a person had larger networks involved in their care according to the family physician, this led to more page views. Having an informal caregiver also contributed positively to ZWIP usage. Also, patients with a higher Barthel index (better ADL) at baseline showed fewer page views compared to the older people with lower Barthel index (worse ADL) at baseline. All patients slightly decreased in their use of ZWIP over time. On a family practice level, having a higher percentage of frail older people in the practice, previously having used electronic consultation methods, and being located in a village (versus a city) led to more active ZWIP usage.

    Also, a major theme addressed in the interviews was facilitators of use. According to the interviewees, a successful older ZWIP user was interested in communicating about health and welfare status and was pro-actively sending messages to providers using ZWIP: “I like to be in control of my own care. I can handle different opinions. And I have several health problems, on which I have questions” [Respondent 22, Patient, on why ZWIP was successful in his case].)

    Importantly, these patients had current, chronic health problems for which they used ZWIP. Patients using ZWIP should have computer skills and should have no cognitive problems, or should have an adequate informal caregiver who can use ZWIP on their behalf: “I can handle it (computer) very well. My mother cannot, so I support her” [Respondent 18, Informal caregiver]. ZWIP was considered a good, quick, and easy communication tool: “I can easily communicate without picking up the phone for everything” [Respondent 10, Informal caregiver].

    Informal caregivers, especially children who were living far away, mentioned that ZWIP made it easy for them to be involved in the care for their vulnerable parent. “It is a triangle, it is about my husband, but it could also support me” [Respondent 19, Informal caregiver] and “Before, you had to wait until the family practitioner came. Now we used ZWIP and the family practitioner was already informed when he came” [Respondent 3, Informal caregiver].

    An important advantage compared to telephone and face-to-face communication was that in ZWIP it is possible to communicate at a time that is convenient for patients and providers: “A small example is that I do not have to take my mother to the family practitioner for everything. I can just ask a question” [Respondent 18, Informal caregiver].

    Some patients mentioned that they used ZWIP because it gave them a sense of safety, being able to communicate with their health care providers and to ask questions in ZWIP: “It is good that I can alarm providers about my father’s situation, rather than them hearing only his story” [Respondent 18, Informal caregiver]. One patient said that communication in ZWIP could replace a consultation, which was convenient for him because his medical condition made it difficult for him to visit his physicians: “Before, you had to wait until the family practitioner came. Now we used ZWIP and the family practitioner was already informed when he came” [Respondent 3, Informal caregiver].

    Another major theme in the interviews was barriers for use. Patients or informal caregivers who did not or hardly used ZWIP often stated that there was no need for ZWIP, since they or the older people they cared for were healthy: “There was no reason for use. My father is actually still very healthy” [Respondent 15, Informal caregiver] and “Why should I communicate if I do not have any questions?” [Respondent 23, Patient].

    Some older people simply had problems logging in but never asked for help, or they reported that support was not sufficient: “I thought it was all very complicated…I could not handle it” [Respondent 9, Informal caregiver] and “A computer is a little impossible for an older person” [Respondent 15, Informal caregiver].

    However, patients did expect an increased usage of such interventions in the future when computer skills are more common among older patients. Other patients did not like the fact that ZWIP was a computer app and were not interested in using it. Some patients feared that the ZWIP would replace face-to-face contact with their health care providers and preferred to phone or visit the family practice: “You do not see your family practitioner anymore. That is a disadvantage… When contact is only digital, I think care becomes rather meager” [Respondent 16, Patient].

    Other patients and informal caregivers would have wanted to use ZWIP but complained that they got no response or only a limited response from health care providers. This was particularly problematic if it was their family practitioner not responding. “We experienced slow reactions from health care providers…So we could not use it for communication” [Respondent 20, Informal caregiver].

    Some of these professionals did not even accept the invitation to participate in the patient’s network. Several patients said that they assumed that their family practitioners were too busy to communicate in ZWIP and so did not try further: “I found it cumbersome. And because I thought: they are never going to make time for that. They are already so busy” [Respondent 6, Informal caregiver].

    Table 3. Association of potential patient-level and family practice/practitioner-level predictors with the number of total page views at baseline and across 2 years of follow-up using hierarchical linear models.
    View this table


    Principal Findings

    Active use of ZWIP as an innovative POHC, specifically developed for frail older people, was limited, despite a small group of very active users. However, in active networks ZWIP was highly appreciated. ZWIP was mainly used for communication. The combination of quantitative and qualitative data revealed a distinct profile of an active ZWIP user: a patient or informal caregiver interested in communicating about their own or their relative’s health and welfare status and sufficiently pro-active to actually stimulate communication in ZWIP. Importantly, these patients had current, chronic health problems that they used ZWIP for and had a lower functional performance (low Barthel scores). Multiple health care providers in a patient’s network (reflected in network size) was also associated with increased ZWIP use.

    Characteristics of Likely Users

    The current study contributes to identifying characteristics of the older people who are likely to use POHCs—either on their own or with additional support—and which older people are unlikely to use a POHC. It seems that additional targeting criteria are useful for the successful implementation of POHCs for frail older people. Whereas some general characteristics may be applicable for all potential users of eHealth interventions, others seem specific for this population of frail older people. In line with previous research [36], impaired health status (reflected in ADL functioning and indirectly in network size) was an important predictor for using a website in older people. This indicates that a minimal level of care need was required. So there needs to be a current reason for intensive contact with health care providers for a POHC to be used. However, other characteristics identified by previous research [37] such as education were not significantly related to ZWIP use. This may be due to the generally low level of education of the older people in the sample compared to younger populations. Additionally, patient age and gender seem to be a common predictor of usage [33]; however, in this study neither was significant. This may be due to the low level of ZWIP’s use. At the same time, the effect of age is partially captured by the health status variable, as shown by a small and significant negative interaction between age and continued usage (analysis on request). Further research is needed to investigate how age influences usage beyond health status.

    Two additional characteristics of successful implementation are worth mentioning. On a family practice level, this study shows that a high percentage of frail older people in the family practice’s patient population led to more ZWIP use. This finding suggests that context effects are also important for usage. Additionally, having an informal caregiver significantly contributed to ZWIP’s increased usage, which may indicate that informal caregivers can overcome problems of (generally) low computer literacy among older people, as suggested by previous research [38,39].

    According to interviewees, future active ZWIP users are interested in managing their own (or their loved one’s) care and experiencing current health problems. Such observations are consistent with major dimensions of the theoretical models of information and communication technology adoption, such as the Unified Theory of Acceptance and Use of Technology model, namely performance expectation, effort expectancy, facilitating conditions, computer experience and voluntariness of use, and social influence [33]. Performance expectancy is defined as the degree to which individuals believe that using the system will help them attain gains in performance. Effort expectancy is the degree of ease associated with the use of the system. Facilitating conditions are defined as the degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system. Computer experience is if patients have used computers before, voluntariness of use is how far patients felt they had a choice in using the system, and social influence is having the feeling that others also use the system and their opinions on the system. Specifically, in terms of performance expectations, patients were more likely to use ZWIP if they expected improved communication with the physician. In terms of effort expectancy, on average they perceived ZWIP use to be easy, although there is a group who find the computer a real barrier to ZWIP use. Patients were also more likely to use ZWIP if they were facilitated in ZWIP usage or had previous computer experience.

    Additionally, ZWIP usage was more likely if patients voluntarily participated in ZWIP, and in terms of social influence, actually got messages. Therefore, it seems that the successful use of patient networks requires providers to be willing to respond to patient messages and actively send messages to patients. Conversely, health care providers who decline invitations or fail to respond to patient messages form a serious barrier for implementation. Further research is needed to quantitatively explore the role of technology acceptance factors, for example, using the Unified Theory of Acceptance and Use of Technology model [33] to guide implementation of POHCs in older populations. However, this first study to identify the characteristics of successful ZWIP participants already allows for a stricter selection of patients who might most easily benefit from such interventions. As long as a large group of frail persons experiences the electronic nature of POHCs as a major barrier to use, the efficacy and effectiveness of POHCs can be better tested in more selective samples [40,41]. Carefully selected older people should be more successful in digital self-management.

    Implications for Implementation and Practice

    Although we have found some examples of frail older patients in their 80s successfully using ZWIP, low computer literacy of older people seems to hamper the implementation efforts. In the Netherlands, only 39% of older people aged 75 years and older report having Internet access [42], whereas in the younger populations this is close to 100%. One option is to give older people computer training. Previous research has shown that training older people to use computers and health websites that they are expected to use can be successful [43]. Therefore, we have attempted train older people in using computers and the ZWIP website [44]. However, older people were not receptive to our chosen method of implementation of receiving trainers in their homes [44]. An alternative is involving the informal caregivers at an earlier stage, as both our qualitative results and quantitative results revealed the importance of having an informal caregiver for continued use. It is likely that the general level of computer literacy in older people will increase over time. However, low computer literacy will remain problematic in disadvantaged groups or very old people for a considerable time and may be a persistent problem for older people suffering cognitive impairment. In future studies, various implementation methods should be compared to raise computer literacy in the older people and to increase the willingness of informal caregivers to participate. In addition, the effect of using eHealth interventions on various components of health—namely physical, social, and mental health—should continue to be explored further.

    Strengths and Limitations

    This study reports a process evaluation of POHC usage in a population of frail older people. Its major strength lies in the mixed methods approach, combining both qualitative and quantitative information on usage, and in its relatively large sample size. Its outcome, a profile of a successful older eHealth user, makes it possible to target likely users. Adequately targeting audiences who will use a new intervention, represents a precondition for studies aiming to show the benefits of such interventions. Thus, these profiles form a valuable contribution to future research and implementation projects of eHealth interventions.

    The study also has some limitations to consider. First, this process evaluation of usage was conducted in the Netherlands within a strong primary care system and so it can be generalized only to countries with a similar system. Additionally, we did not continuously measure the health status of individuals, which was shown qualitatively as a main driver of usage, and this should be explored in further research. Finally, while we explored patients’ levels of computer literacy, access to computers, and user-friendliness of ZWIP for the individual older person in the interviews, we omitted a quantitative assessment of these issues and this should be pursued further during evaluation of other applications.


    The current evaluation of use of the ZWIP, a personal online health community for frail older people, revealed important predictors of usage of eHealth interventions in older populations. Frail older patients with poor health status reflected in functional problems, with at least 5-6 providers involved perceived the most benefit and therefore used ZWIP more actively. Sufficient computer skills in either the patient or the informal caregiver and an interest to play an active role in their own care were essential. Therefore, during implementation of POHCs, and probably other eHealth interventions, informal caregivers should be involved from the start of the project. Profiling successful users can facilitate more effective targeting of frail older patients for implementation of eHealth interventions in the near future.


    We would like to thank Wilma Derksen-Driessen, Emile ter Horst Mark Kuster, Charlotte Neger, Jean Nielen, Joep Scheltinga, and Leontien van Nieuwenhuijzen for their assistance in the development and implementation of the program. This study was supported by the Netherlands Organisation for Health Research and Development (ZonMw) project number 311050201. The funder had no role in conducting the study, analysis of the data, or during the publication of the manuscript.

    Authors' Contributions

    The authors contributed to the manuscript as follows: PM drafted the manuscript; HS, MP, SHMR contributed to acquisition of data; PM did the statistical analysis of data; PM, MP, SHMR, MGMOR, RJFM interpreted the data. All authors contributed to the study design, revised the manuscript for important intellectual content, and approved the manuscript. MGMOR and RJFM supervised the study and obtained funding.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    ZWIP usage.

    WMV File (Windows Media Video), 15MB

    Multimedia Appendix 2

    Interview questions.

    PDF File (Adobe PDF File), 4KB


    1. Rockwood K, Mitnitski A. Frailty in relation to the accumulation of deficits. J Gerontol A Biol Sci Med Sci 2007 Jul;62(7):722-727. [Medline]
    2. Robben SH, Huisjes M, van Achterberg T, Zuidema SU, Olde Rikkert MG, Schers HJ, ZOWEL NN Study Group. Filling the Gaps in a Fragmented Health Care System: Development of the Health and Welfare Information Portal (ZWIP). JMIR Res Protoc 2012;1(2):e10 [FREE Full text] [CrossRef] [Medline]
    3. Vedel I, Akhlaghpour S, Vaghefi I, Bergman H, Lapointe L. Health information technologies in geriatrics and gerontology: a mixed systematic review. J Am Med Inform Assoc 2013;20(6):1109-1119 [FREE Full text] [CrossRef] [Medline]
    4. van den Berg N, Schumann M, Kraft K, Hoffmann W. Telemedicine and telecare for older patients--a systematic review. Maturitas 2012 Oct;73(2):94-114. [CrossRef] [Medline]
    5. van der Eijk M, Faber MJ, Aarts JW, Kremer JA, Munneke M, Bloem BR. Using online health communities to deliver patient-centered care to people with chronic conditions. J Med Internet Res 2013;15(6):e115 [FREE Full text] [CrossRef] [Medline]
    6. Demiris G. The diffusion of virtual communities in health care: concepts and challenges. Patient Educ Couns 2006 Aug;62(2):178-188. [CrossRef] [Medline]
    7. Makai P, Perry M, Robben S, Schers HJ, Heinen MM, Olde Rikkert MG, et al. Evaluation of an eHealth intervention in chronic care for frail older people: why adherence is the first target. J Med Internet Res 2014;16(6):e156 [FREE Full text] [CrossRef] [Medline]
    8. Kim EH, Stolyar A, Lober WB, Herbaugh AL, Shinstrom SE, Zierler BK, et al. Challenges to using an electronic personal health record by a low-income elderly population. J Med Internet Res 2009;11(4):e44 [FREE Full text] [CrossRef] [Medline]
    9. DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris TG, Jha A, et al. Electronic health records in ambulatory care--a national survey of physicians. N Engl J Med 2008 Jul 3;359(1):50-60. [CrossRef] [Medline]
    10. Jha AK, Doolan D, Grandt D, Scott T, Bates DW. The use of health information technology in seven nations. Int J Med Inform 2008 Dec;77(12):848-854. [CrossRef] [Medline]
    11. Jha AK, DesRoches CM, Campbell EG, Donelan K, Rao SR, Ferris TG, et al. Use of electronic health records in U.S. hospitals. N Engl J Med 2009 Apr 16;360(16):1628-1638. [CrossRef] [Medline]
    12. Jha AK, Bates DW, Jenter C, Orav EJ, Zheng J, Cleary P, et al. Electronic health records: use, barriers and satisfaction among physicians who care for black and Hispanic patients. J Eval Clin Pract 2009 Feb;15(1):158-163. [CrossRef] [Medline]
    13. Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R. Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. hospitals. Health Aff (Millwood) 2004;23(4):184-190 [FREE Full text] [Medline]
    14. Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ 2012 May 1;90(5):357-364 [FREE Full text] [CrossRef] [Medline]
    15. Verhoeven F, van Gemert-Pijnen L, Dijkstra K, Nijland N, Seydel E, Steehouder M. The contribution of teleconsultation and videoconferencing to diabetes care: a systematic literature review. J Med Internet Res 2007;9(5):e37 [FREE Full text] [CrossRef] [Medline]
    16. Paré G, Moqadem K, Pineau G, St-Hilaire C. Clinical effects of home telemonitoring in the context of diabetes, asthma, heart failure and hypertension: a systematic review. J Med Internet Res 2010;12(2):e21 [FREE Full text] [CrossRef] [Medline]
    17. Cartwright M, Hirani SP, Rixon L, Beynon M, Doll H, Bower P, Whole Systems Demonstrator Evaluation Team. Effect of telehealth on quality of life and psychological outcomes over 12 months (Whole Systems Demonstrator telehealth questionnaire study): nested study of patient reported outcomes in a pragmatic, cluster randomised controlled trial. BMJ 2013;346:f653 [FREE Full text] [Medline]
    18. van Kempen JA, Schers HJ, Jacobs A, Zuidema SU, Ruikes F, Robben SH, et al. Development of an instrument for the identification of frail older people as a target population for integrated care. Br J Gen Pract 2013 Mar;63(608):e225-e231 [FREE Full text] [CrossRef] [Medline]
    19. Olde-Rikkert MG, Long JF, Philp I. Development and evidence base of a new efficient assessment instrument for international use by nurses in community settings with older people. Int J Nurs Stud 2013 Sep;50(9):1180-1183. [CrossRef] [Medline]
    20. Extermann M, Aapro M, Bernabei R, Cohen HJ, Droz JP, Lichtman S, Task Force on CGA of the International Society of Geriatric Oncology. Use of comprehensive geriatric assessment in older cancer patients: recommendations from the task force on CGA of the International Society of Geriatric Oncology (SIOG). Crit Rev Oncol Hematol 2005 Sep;55(3):241-252. [CrossRef] [Medline]
    21. Creswell JW, Klassen A, Plano Clark V, Smith KC. National Institutes of Health. Bethesda (Maryland); 2011. Best practices for mixed methods research in the health sciences   URL: [accessed 2014-11-18] [WebCite Cache]
    22. Lingard L, Albert M, Levinson W. Grounded theory, mixed methods, and action research. BMJ 2008;337:a567. [Medline]
    23. Wade DT, Collin C. The Barthel ADL Index: a standard measure of physical disability? Int Disabil Stud 1988;10(2):64-67. [Medline]
    24. Collin C, Wade DT, Davies S, Horne V. The Barthel ADL Index: a reliability study. Int Disabil Stud 1988;10(2):61-63. [Medline]
    25. Katz S. Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. J Am Geriatr Soc 1983 Dec;31(12):721-727. [Medline]
    26. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992 Jun;30(6):473-483. [Medline]
    27. Beckstead JW. Using hierarchical cluster analysis in nursing research. West J Nurs Res 2002 Apr;24(3):307-319. [Medline]
    28. Ward JH. Hierarchical Grouping to Optimize an Objective Function. Journal of the American Statistical Association 1963 Mar;58(301):236-244. [CrossRef]
    29. Calinski T, Harabasz J. A dendrite method for cluster analysis. Comm. in Stats. - Theory & Methods 1974;3(1):1-27. [CrossRef]
    30. Singer JD. Using SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and Individual Growth Models. Journal of Educational and Behavioral Statistics 1998 Jan 01;23(4):323-355. [CrossRef]
    31. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ 2001 May 5;322(7294):1115-1117 [FREE Full text] [Medline]
    32. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005 Nov;15(9):1277-1288. [CrossRef] [Medline]
    33. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: Toward a unified view. MIS quarterly 2003:425.
    34. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005 Nov;15(9):1277-1288. [CrossRef] [Medline]
    35. Creswell JW, Fetters MD, Ivankova NV. Designing a mixed methods study in primary care. Ann Fam Med 2004;2(1):7-12 [FREE Full text] [Medline]
    36. Chou WY, Hunt YM, Beckjord EB, Moser RP, Hesse BW. Social media use in the United States: implications for health communication. J Med Internet Res 2009;11(4):e48 [FREE Full text] [CrossRef] [Medline]
    37. Kakkar A, Jacobson BC. Failure of an Internet-based health care intervention for colonoscopy preparation: a caveat for investigators. JAMA Intern Med 2013 Jul 22;173(14):1374-1376. [CrossRef] [Medline]
    38. Sadasivam RS, Kinney RL, Lemon SC, Shimada SL, Allison JJ, Houston TK. Internet health information seeking is a team sport: analysis of the Pew Internet Survey. Int J Med Inform 2013 Mar;82(3):193-200. [CrossRef] [Medline]
    39. Mazor KM, Williams AE, Roblin DW, Gaglio B, Cutrona SL, Costanza ME, et al. Health literacy and pap testing in insured women. J Cancer Educ 2014 Dec;29(4):698-701. [CrossRef] [Medline]
    40. Bates DW, Wells S. Personal health records and health care utilization. JAMA 2012 Nov 21;308(19):2034-2036. [CrossRef] [Medline]
    41. Makai P, Melis RJ, Olde-Rikkert MG. Technical difficulties and evaluating e-health interventions. JAMA Intern Med 2014 Feb 1;174(2):304-305. [CrossRef] [Medline]
    42. Statistics Netherlands. Rapid increase of internet usage among older people. 2013.   URL: http:/​/www.​​en-GB/​menu/​themas/​vrije-tijd-cultuur/​publicaties/​artikelen/​archief/​2013/​2013-4005-wm.​htm [accessed 2014-11-18] [WebCite Cache]
    43. Xie B, Bugg JM. Public library computer training for older adults to access high-quality Internet health information. Libr Inf Sci Res 2009 Sep 1;31(3):155 [FREE Full text] [CrossRef] [Medline]
    44. Robben SH, Perry M, Huisjes M, van Nieuwenhuijzen L, Schers HJ, van Weel C, et al. Implementation of an innovative web-based conference table for community-dwelling frail older people, their informal caregivers and professionals: a process evaluation. BMC Health Serv Res 2012;12:251 [FREE Full text] [CrossRef] [Medline]


    ADL: Activities of Daily Living
    Easycare-TOS: Two-step Older person Screening
    IADL: Instrumental Activities of Daily Living
    POHC: Personal Online Health Community
    SF-36: Short Form-36 Health Survey
    ZWIP: Zorg en Welzijn Informatie Portaal (Health and Welfare portal)

    Edited by G Eysenbach; submitted 17.06.14; peer-reviewed by R Greysen, T Luger2; comments to author 07.08.14; revised version received 04.09.14; accepted 20.10.14; published 17.12.14

    ©Peter Makai, Marieke Perry, Sarah HM Robben, Henk Schers, Maud Heinen, Marcel GM Olde Rikkert, René JF Melis. Originally published in the Journal of Medical Internet Research (, 17.12.2014.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.