A detailed protocol [18] and 2 articles describing MT4C and its evaluation have been published elsewhere [15,19]; thus, only details relevant to this secondary analysis are provided herein. Similar to the pragmatic trial, this study utilized a mixed methods comparative design. The data reported here focus on family carers allocated to the treatment group (N=101). Qualitative data from the interviews and quantitative data from the treatment group collected during the original study were integrated in the results stage. Qualitative data were used to understand the quantitative results.

The primary study received ethical approval from the University of Alberta Health Research Ethics Board (No. Pro00048721) and the Hamilton Integrated Research Ethics Board (No. 15-309). The initial ethics application included the ability to conduct a secondary analysis of the data.

Family carers were invited to participate if they were over the age of 18 years and were caring for a person aged 65 years or older living with ADRD and MCC in the community. In addition, they needed to have a valid email address and access to a computer. Family carers were recruited through multiple community organizations including Alzheimer Society branches in each province and advertisements in local community newspapers in Alberta. If they met the eligibility criteria, they were asked to contact the researchers. Participants were randomly assigned to a treatment or control group using stratified permuted block randomization. Different consent forms (one for the treatment group and one for the control group) were used to blind participants regarding their group assignment.

Once consent was obtained, trained data collectors collected all data (baseline, one month, and three months) via telephone and entered it into REDCap, a secure, password-protected Web-based data collection service, offered at the University of Alberta. Data collection occurred from June 2015 to April 2017 and is reported in more detail in the study protocol [18]. Data collection procedures for the treatment group and the number of participants at each period are presented in Figure 1.

Following baseline interviews, participants in the treatment group received free, password-protected access to MT4C for three months. Research assistants, using a standardized script, instructed the participants to access MT4C at their convenience on a computer, tablet, or smartphone. A follow-up email was sent to participants in the treatment group with instructions on how to access the site and their login information. After logging in, the first page provided instructions on how to use MT4C. Each Web page also contained a menu outlining the sections that comprised MT4C: (1) about me; (2) common changes to expect; (3) frequently asked questions; (4) resources; (5) important health information; and (6) calendar. MT4C also provided options to add formatted text, photos, and PDF files in certain sections. All information entered by participants was treated confidentially and was not accessible to the study team. Participants also received an electronic copy of the Alzheimer Society’s The Progression of Alzheimer’s Disease booklet [20], a copy of the study questionnaires, and the MT4C toolkit checklist intended for participants to record their use of the MT4C site. During the one-month interview, trained data collectors encouraged nonusers to use MT4C. No changes or alterations were made to MT4C during the study.

Data were entered in SPSS version 24 (IBM) and checked for accuracy by a trained research assistant. Before data analysis, participants were divided into 2 groups: (1) participants who used MT4C at least once over three months and (2) participants who did not use MT4C within the three-month period. Use of MT4C was captured using a dichotomous variable, where 1=used MT4C at least once during the three-month intervention period and 0=did not use MT4C during the three-month intervention period.

A total of 101 participants were allocated to the treatment group at baseline. Following baseline measures, 9 participants withdrew and the remaining 92 participants received instructions on how to access MT4C. Figure 1 illustrates the number of persons at baseline and three months. The mean age of all participants in the treatment group was 63.5 years (SD 12.0), and they had been carers for an average of 4.1 years (SD 3.9). The majority of participants were female (73/92, 79%), white (84/92, 91%), were living with a person with ADRD (63/92, 62%), and were the spouse of a person with ADRD (48/92, 52%). No statistically significant differences were found in the demographic characteristics of users and nonusers (Table 1).

The means and SDs of the outcome measures (HHI, GSES, MCS, and PCS for each group at baseline and three months) are presented in Table 2.

Table 3 provides the ANCOVA results for each outcome. The group variable (users and nonusers) was significant for the GSES outcome (P=.003), indicating that the use of MT4C during the three-month period was associated with an increase in GSES from baseline to three months. The use of MT4C was not associated with significant differences in the other outcomes.

Table 4 provides the GEE model results and shows that the use of MT4C was associated with an increase in GSES over three months (P=.048). The use of MT4C was not associated with significant changes in the other outcomes.

Table 2

Mean and SD of outcomes at baseline and three months for users and nonusers.

Reasons for nonuse of MT4C reported in the qualitative data included the following: caregiving demands; problems accessing and navigating the site; and preference for paper or in-person contact. Participants reported being consumed with the role of caregiving and as a result did not have enough time to use MT4C. As one participant said: “...and I got to admit that it was, uh, something that, uh, I didn’t go onto too much, just strictly because of all the other things that were—were going on this past month.” Those who were able to find a little bit of time to look at MT4C found they were quickly distracted by the care recipient and ultimately did not use it. One participant described having to stop using MT4C to tend to her husband: “Well, um, I just finished reading it, and—and—and, then, I had to go off because I had to go help my husband.”

Caregiving demands also resulted in nonuser participants feeling stressed. As one participant said: “I’m extremely stressed with taking care of my wife, and so I lost the email with login instructions.” Another indicated that the lack of energy was a factor “...[I] work full time early morning to late evening...and at the end of the day, I don’t have the energy or time to go on the computer.”

Problems accessing MT4C were related to poor internet connections, computer literacy, and difficulties navigating the site. Nonusers who lived in rural areas reported poor internet connections: “...my internet connection at home is poor—I live in a rural area.” Several nonusers described their lack of experience with computers (computer literacy): “No, it—it’s, uh, as far as the computer is concerned, it’s the—the operator of it that’s at fault.”; another said: “Um, well, I get frustrated at myself when, you know, I’m working on the website...”

In terms of difficulty navigating the site, participants described forgetting the link to log in and difficulty printing instructions for the site. A participant described her frustration with not being able to find where she had previously been working on the site after being interrupted by her husband, “he kept interrupting me. Then, I couldn’t find where I left off to continue...” Caregiving demands coupled with navigating the site were the reason this participant did not use MT4C.

Nonusers also seemed to have a preference for access to hard copy or paper format of MT4C and interaction with other carers. As a participant said, “...Sometimes, you actually have to have something printed in front of you, uh, and I’m better off—I’m better with paper. In some instances, to sit and reflect, I’m not really good at what—I’m not really one of those people who can do it all on-line.” This participant described his lack of experience with working on the Web, but also suggested his preference was for paper. Another participant suggested a preference for social interaction rather than Web-based tools: “I think—I think I know—and this is [chuckles]—this isn’t specific to this Toolkit, but it sort of relates to it: um, I think I’m the kind of person who gets a lot more out of, you know, actual social interaction around something.”

The reasons for nonuse of MT4C, although reported as separate issues, appeared to be interrelated. For example, the lack of computer literacy meant it took more time to use MT4C, but with caregiving demands, less time was available. Furthermore, participants who are not computer literate became frustrated and thus preferred a paper format.

The main findings of this study were a statistically significant difference between the users and nonusers of MT4C with regard to general self-efficacy (the confidence in their ability to deal with difficult situations). General self-efficacy significantly increased in the user group.

Other studies have found that general self-efficacy has a significant positive relationship with the quality of life of family carers of people with chronic illness [28]. For family carers of persons with dementia, self-efficacy has been found to have a significant relationship with health outcomes such as mental health [29,30] and, in particular, is negatively correlated with depression [31,32]. In the larger study, hope was found to be positively associated with general self-efficacy [13]. When the hope of participants increased, so did their confidence in their ability to deal with difficult situations. However, in the larger study using the intent-to-treat analysis, which included nonusers, general self-efficacy was not found to be significantly different in the treatment versus the control group, but hope was. It was important to further examine differences between users and nonusers in the treatment group, as the finding that self-efficacy was higher in users versus nonusers supports the intervention model in which MT4C has the potential to increase quality of life by increasing self-efficacy.

The qualitative data suggested that caregiving demands, accessibility to the site, and preference for a paper version or face-to-face interaction were barriers to use for nonuser participants. Caregiving demands with subsequent family carer lack of energy and feelings of stress are consistent with the findings from the qualitative data from all participants in the larger study [19]. Quality of life scores at each time period for users and nonusers were not significantly different; however, whether the nonuser group experienced a more pronounced lack of energy and higher levels of stress than those in the user group is unclear. Future research should potentially also measure fatigue and stress as possible barriers to the use of Web-based interventions.

Poor connectivity to the internet was described as a barrier to use of MT4C by nonuser participants. Web-based interventions have been considered to be of benefit particularly for rural populations because of considerations related to accessibility [33]. However, poor connectivity to the internet, particularly for persons in rural areas in Canada, is a barrier to the use of any Web-based intervention [34,35]. Poor connectivity should be a concern for any research with Web-based interventions and possibly an exclusion criterion for participants in efficacy and effectiveness trials.

Computer literacy (ie, the ability to use computers and related technology efficiently) appeared to be one of the barriers to using MT4C. Inclusion criteria for the study included access to a computer and an email address. However, in this study, a measure of computer literacy was not used. Park et al [36], following an integrated review of health-related internet use of family carers of children, suggested that Web-based interventions should also include educational programs to increase computer literacy. Although MT4C was previously determined to be easy to use, an additional tell me more feature could be embedded into the program to assist carers who have low computer literacy.

What is unclear from our study is if access issues and computer literacy resulted in some participant preferences for a paper format and/ or face-to-face supportive interactions. Moreover, the nonusers referred in their comments to in-person interaction with other carers not Web-based interaction with other carers. When interaction with other family carers was added to a Web-based intervention, caregiving demand and computer literacy were also found to have an impact on the perceived benefit of a Web-based intervention [37]. This suggests that Web-based interventions to support family carers of persons with ADRD and MCC should not be the only format for support, but opportunities to use a paper version of MT4C and in-person face-to-face interactions are also important.

This study was a secondary analysis; thus, follow-up interviews with nonuser participants were not conducted. Follow-up interviews would have been completed to further explore participants’ reasons for nonuse and to answer questions about the relationship between computer literacy and their preferences for a paper format. Low computer literacy and poor connectivity were also not considered as exclusion criteria for the study, possibly influencing the results.

Importantly, this comparison involves a sample that was not randomly assigned, thus limiting the generalizability of the study. Although there were no statistically significant differences in the demographic characteristics between users and nonusers, there may be a potential imbalance between the groups based on unmeasured contextual characteristics. The findings contribute to the developing model of the intervention; however, in future studies, potential mediators and moderators should be identified and their influence on the outcomes of the intervention should be evaluated.

The findings of this study reflect how comparisons of users and nonusers in Web-based intervention studies can improve Web-based interventions and the design of future studies. The statistically significant higher levels of general self-efficacy (or the confidence in the ability to deal with difficult situations) in users of MT4C is an important finding. Family carers of persons with dementia have reported significantly less self-efficacy than carers of persons without dementia [38]. As such, MT4C can potentially benefit family carers who are willing and able to use Web-based interventions. More research is needed to determine if adding an educational program for computer literacy may assist more family carers to access this Web-based intervention. In addition, future research should explore the use of MT4C in carers with diverse cultural backgrounds and languages.