To answer the research questions, a 3-arm RCT was performed with repeated measurements at 3 time points. The 3 intervention arms all focused on helping family caregivers deal with behavior changes in their relative with dementia but varied in the number of elements involved. The intervention arms are referred to as the major, medium, and minor intervention arms. The intervention arms are described elsewhere in more detail [19]. The major intervention arm consisted of the following:

The medium intervention arm consisted only of the online videos and e-bulletins, and the minor intervention arm consisted only of the e-bulletins. For more details, the readers are referred to the full intervention protocol [20].

Family caregivers were eligible to participate in the study if they were at least 18 years old, were a partner or relative of a person diagnosed with dementia who lives at home, had contact with the person with dementia at least once a week, had access to the internet, and gave online consent. Family caregivers were recruited via the Dutch Alzheimer Society’s panel, the Dutch Alzheimer Society’s online forum (with 7000 monthly visitors), the Dementie.nl website [21], and the Dutch Alzheimer Society’s social media accounts (Twitter and Facebook). Details of the recruitment procedure have been described elsewhere [19].

After online consent was given (see the study protocol for more detail [19]), family caregivers were randomly allocated by a researcher (JGH) to 1 of the 3 intervention arms using a randomization schedule. Block randomization was applied to achieve an equal likelihood of the participant being allocated to each of the 3 intervention arms [22]. An independent epidemiologist prepared the randomization schedule using several block sizes of 6 and 9.

Participants could not be blinded as it is impossible to blind participants to the sort of eHealth intervention they are receiving [23].

In this study, we expected that (1) both the major and medium intervention arms would lead to a greater improvement in self-efficacy than the minor intervention arm and that (2) the major intervention arm would show larger improvements in self-efficacy than the medium intervention arm.

In another study, large effect sizes were found for self-efficacy in family caregivers with dementia [24]. Based on a difference of 0.8 standard deviation units between the groups and assuming a significance level of 5%, a power of 80%, and correlation of .60 between the two repeated measures, the number of subjects needed per group was 20. Taking into account a dropout rate of 20%, 24 participants per group were needed.

Another consideration was that the specialist dementia nurses had limited previous experience in providing self-management support through email contacts. We therefore expected a learning curve for the dementia nurses during the study, which might also have had consequences for the measured effects on family caregivers. Following the randomization schedule, one additional block of 9 participants (3 in each group) was added to the sample so that we could take a brief learning curve into account. This brought the total number of participants that had to be recruited to 81.

Measurements were performed at 3 points in time: (T0) baseline assessment, (T1) 6 weeks after the baseline assessment, and (T2) 12 weeks after the baseline assessment. Measurements were done by online questionnaires administered to the participating family caregivers through an email link. After 1 and 2 weeks, participants were reminded (if needed) to complete the questionnaires.

The primary outcome variable (self-efficacy) was measured using the Trust in Our Own Abilities (TRUST) instrument, a questionnaire in Dutch. The questionnaire had been used previously to measure self-efficacy in family caregivers of people with dementia living at home [25]. The TRUST questionnaire has 32 items divided into 3 subscales: solution orientation (8 items), resilience (15 items), and proactive competence (9 items). For this study, one item from the original 37-item TRUST questionnaire was added as this item reflected the main goal of this intervention. This item was queried as “How well can you, in your own opinion, deal with changed behavior of your relative, such as aggression, apathy, and dependence?” (translated from Dutch). Since the TRUST questionnaire is quite new and has only been validated and tested with pilot data, a principal component analysis was performed. A total of 33 items were tested in a principal component analysis. All 33 items were loading on the same factor. However, 4 of the 33 items were not loading strongly enough (cutoff point <0.4) [26]. When these items were dropped, the Cronbach alpha for our sample was .925. Only the revised sum score (29 items) will therefore be studied. Items ranged from 0=not at all to 4=very good). The higher the score, the greater the perceived competence in caring for someone with dementia [25].

The first secondary outcome variable was the presence and reaction scores for mood and behavior problems, measured using the Dutch version of the Revised Memory and Behavioral Problem Checklist (RMBPC) [27,28]. The RMBPC is a self-assessment questionnaire that can be broken down into scales for disruptive behavior (8 items), depression (9 items), and memory-related problems (7 items). Overall reliability for this scale is .84 for patient behavior and .90 for caregiver reaction [27].

For this study, only disruptive behavior will be studied as this was the outcome of interest. Family caregivers were asked to rate the occurrence of specific behavior on a scale from 0 to 4 (0=never, 1=rarely, 2=regularly, 3=often, 4=always) and parallel their reaction scores for the degree of distress (0=not upset, 1=not very upset, 2=quite upset, 3=extremely upset).

The mean scores of the occurrence of behavior and family caregivers’ reaction to these problems were calculated. For behaviors that did not occur, a reaction score of 0 (not upset) was assigned [29].

A second secondary outcome variable concerned the positive and negative aspects of the relationship between the person with dementia and the family caregiver. This was measured by the Dyadic Relationship Scale (DRS). The family caregiver version has 11 items in 2 subscales: dyadic strain (5 items) and positive dyadic interaction (6 items). Family caregivers were asked to rate the separate items on a 4-point scale (1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree). Overall reliability for this scale is .89 for negative dyadic strain and .85 for positive dyadic interaction [30].

All data were analyzed using SPSS Statistics version 22.0 (IBM Corp). Mixed-model analyses were carried out to compare primary and secondary outcomes between the major and minor intervention arm and between the medium and minor intervention arm over time and at T1 and T2. Mixed-model analyses were performed to take into account the correlation between the 2 repeated measurements within the subject (T1 and T2). To obtain the intervention effect at 2 different time points, time and interaction between intervention and time were added to the model. All mixed-model analyses were adjusted for the baseline value of the particular outcome. In addition to crude effects, effects adjusted for gender, type of relationship, appearance of first symptoms, education level, and shared caregiving were also estimated.

The study was approved by the VU University Medical Center’s Medical Ethics Committee (reference 2016.559). It had no objections to the study. All participants were required to give their informed consent for participation via an online informed consent form. Only the research team members had access to the data. Agreements about how to archive, share, and store data were signed by the organizations responsible for collecting the data.

A total of 158 family caregivers expressed interest in participating in the study. After sending an information letter, the first 81 caregivers who signed the online informed consent form and completed the baseline assessment were included.

After completing the baseline questionnaire, participants were randomly allocated to the major (27), medium (27), or minor (27) intervention arms following the block randomization schedule (Figure 1) [31]. A total of 86% (70/81) of family caregivers completed the T1 assessment (6 weeks after baseline), and 82% (66/81) of family caregivers completed the T2 assessment (12 weeks after baseline).

Baseline data for the caregivers included are listed in Table 1. At baseline, family caregivers were on average aged 56.5 (SD 12.5) years (range 23-80 years), primarily female (71/81, 88%), and half of them had completed a professional or academic degree (40/81, 49%). The relatives with dementia they were caring for were mostly their mother or father (or a parent-in-law) (46/81, 57%) or their partner (32/81, 40%). The individuals with dementia were on average aged 75.1 (SD 9.9) years (range 49-96 years) and more often male (42/81, 52%), with Alzheimer disease being the most prevalent form of dementia (47/81, 57%). In most cases, the first symptoms of dementia had appeared 4 years or more previously (42/81, 52%). Behaviors that family caregivers had the most difficulty dealing with were dependent (22/81, 27%) and masking behavior (19/81, 24%). At baseline, most family caregivers stated that they were somewhat (35/81, 43%) or significantly (31/81, 38%) burdened by the care for their relative with dementia.

The initial analyses were performed without the first randomized 9 caregivers (who were the learning curve block). These initial analyses among 72 family caregivers revealed no differences with analyses of data for the overall group of 81 family caregivers. The final analyses were therefore conducted on all 81 randomized family caregivers. Multimedia Appendix 1 and 2 show the results of the mixed-model analyses.

Figure 2 shows the observed mean scores for the sum score of the TRUST questionnaire. In the mixed-model analyses, the major intervention (involving personal email contacts as well as videos and e-bulletins) did not show significant differences in self-efficacy in both the crude and adjusted analyses compared with the minor intervention arm. Also, no statistical differences were found between the medium intervention (involving videos and e-bulletins) and minor intervention (only involving e-bulletins) in the crude analyses.

However, the medium intervention unexpectedly showed a negative trend over time in the adjusted analyses (difference –4.21, P=.09) and at T1 (difference –4.71, P=.07) compared with the minor intervention involving e-bulletins only.

Figure 3 shows the observed mean scores for behavior changes in the person with dementia as reported by the family caregivers. Figure 4 shows the observed mean scores for family caregivers’ reaction scores for disruptive behavior (disruption subscale of the RMBPC questionnaire). No statistical differences were found in the crude and adjusted analyses between the major and minor intervention arms or between the medium and minor intervention arms regarding the occurrence of behavior changes.

However, statistical differences were found between the major and minor intervention arms in the adjusted analyses at T1 for the family caregivers’ reaction scores for disruptive behavior (difference 2.02, P=.05).

Figures 5 and 6 display the observed mean scores for the DRS questionnaire subscales Strain and Interaction. No statistical differences were found in the quality of the relationship in both the crude and adjusted analyses between the major and minor intervention arms and the medium and minor intervention arms at all measurements (over time, at T1 and T2).

Online self-management support involving email contacts with a specialist dementia nurse, videos, and e-bulletins showed no significant difference in family caregivers’ self-efficacy compared with online interventions not involving personal email contacts. Furthermore, no measurable improvements could be found for the medium intervention involving online videos and e-bulletins compared with the minor intervention only involving e-bulletins.

In addition, no differences were found between the online intervention arms for the quality of the relationship between the person with dementia and the family caregiver and the occurrence of behavior changes. These results are contrary to our expectation that family caregivers who received email support would be better assisted in dealing with and responding to changes in behavior and would therefore improve in terms of self-efficacy. We expected that increased self-efficacy and better response of the family caregiver would also have an effect on the person with dementia and would therefore result in less strain on the relationship, better interaction, and n decrease in the occurrence of behavior changes. However, as no effect on self-efficacy was found, this could also explain why no effect could be detected on the secondary outcomes (quality of the relationship and the occurrence of behavior changes) in this study.

Moreover, the medium arm (consisting of video and e-bulletins) showed a negative trend in family caregivers’ self-efficacy over time and shortly after the intervention (at T1). One possible explanation may be that the online videos made family caregivers more aware of how they were dealing with behavioral changes of their relative with dementia. This understanding—obtained from watching the online videos—may have influenced their confidence in their ability to successfully influence behavioral changes. This only seems to affect family caregivers at the moment of watching the video (6 weeks after baseline) and did not remain after a longer period of time (12 weeks after baseline).

This negative trend regarding family caregivers’ self-efficacy was not observed in the major arm, even though those participants shared the same experience of the online videos with the medium arm. Perhaps the personal email contacts with the nurse in the major arm were enough to offset a negative effect of increased awareness through the videos but not enough to have a positive effect on the measured self-efficacy.

An explanation for the lack of improvement in self-efficacy could be that family caregivers were not able to translate the information and advice to their personal situations [7] despite the fact that in the major intervention arm, the dementia nurses tried to tailor their email contacts to the individual situation of the family caregiver. Also, the mean scores at baseline for self-efficacy, behavior changes, and relationships were already quite good. As a result, there might have been less room for improvements.

Contrary to our expectations, it was found that family caregivers in the major intervention arm were significantly more distressed at T1 by the disruptive behavior of their relatives with dementia than family caregivers who only received e-bulletins. An explanation for this can be that, initially, a more intensive and major intervention (involving personal email contacts, videos, and ebulletins) sharpened caregivers’ focus on behavioral changes in their relative with dementia. This initially might have increased awareness, which may have led to an increased report of distress shortly after the intervention at T1. However, there was no statistical difference between these two groups at T2, a more distance time point.

Along with the RCT presented in this paper, a process evaluation was carried out [32]. The process evaluation showed that the personal contacts with the nurse were highly valued and believed to add value to the online videos and e-bulletins. Nonetheless, these qualitative results were not reflected in the quantitative results in this paper.

The process evaluation also gave some additional explanations for the unexpected results in the RCT. First, the process evaluation showed variation in the extent to which family caregivers made use of the various elements. Of the family caregivers in question, 78% used the opportunity of having email contacts and 80% clicked on the links to one or more videos but just 37% of all family caregivers clicked on the links of at least one e-bulletin. Also, the use of email contacts, videos, and/or e-bulletins varied considerably within in each group. Therefore, the distinction between the 3 intervention arms became less, which makes it less likely to find statistically significant differences between the intervention arms. Low use rates and differences in the use of online interventions are known problems [33,34] that could explain why no positive effects were found in this study.

Second, both family caregivers and nurses mentioned that the email contacts helped family caregivers share their stories about their experiences with the changing behavior of their relative with dementia. The email contacts seemed therefore less focused on finding ways to deal with behavioral changes. Although receiving appreciation and acknowledgment is essential for family caregivers [35], this could explain why our study found no effects on self-efficacy, measured behavior, or quality of the relationship.

Last, positive effects could be left out because the participants already knew a lot about dementia and how to deal with behavioral changes of their relative. According to the dementia nurses, the participants involved were mainly family caregivers who were already consciously engaged in collecting information about dementia. These family caregivers all had internet access and were often relatively young and well educated. This group had previously gained information and advice about coping with behavioral changes, which might explain the lack of positive effects on self-efficacy.

Based on the findings of the process evaluation [32], we have 2 recommendations for future use of the intervention. First, we recommend that nurses are instructed more explicitly and made more aware of the importance of the integrated use of the various elements (email contacts, videos, and e-bulletin) in the interventions. Second, for future use the intervention could involve more email contacts.

Several strengths of this study can be noted. First, the online component of this study helped provide accessible and tailored support for family caregivers. Caregivers could participate nationwide and use the online assistance at times that suited them. Second, selective dropout was reduced by using a mixed-model analysis that also included incomplete cases (ie, participants who did not complete the online questionnaire either at the 6- or 12-week follow-up). Finally, selection bias was reduced by using a prepared randomization schedule to randomly allocate family caregivers to 1 of the 3 intervention arms [22].

However, some limitations of this study are worth mentioning. First, in the power calculation, we had estimated a difference of 0.8 between the intervention arms to detect a significant effect of the major self-management support intervention compared with the other intervention arms. The estimated difference proved to have been an overestimate. The small sample size might therefore have played a part in the null findings for our hypothesis that the major intervention arm would have a greater effect on self-efficacy than the other intervention arms. We acknowledge that our study may have been underpowered for detecting an effect of the online self-management support intervention. For future studies, larger studies may be required to establish the effectiveness of online self-management support interventions [36].

Second, due to the small sample size, we were unable to determine the effects on participants who actually used the intervention components. Instead, data of all included participants were analyzed. Future research should focus on which intervention components best fit specific family caregivers. It is important to determine the family caregivers who will benefit the most from additional online assistance in order to provide tailored, personalized support. This will be more cost effective, allowing nurses’ support to be offered to the people who need it the most.

The online self-management support intervention involving email contacts did not lead to positive effects compared with online interventions without personal email contacts. Furthermore, the medium intervention involving online videos and e-bulletins showed no statistical improvements compared with the minor intervention involving e-bulletins only. To come to more definitive conclusions, future research involving extra efforts to achieve high use rates is required.