Dementia is a syndrome that causes deterioration in cognitive functioning. It affects memory, thinking, orientation, comprehension, learning capacity, language, and judgment, and is often accompanied by deterioration of emotional control, social behavior, or motivation. The impairment is severe enough to interfere with daily life. Dementia is caused by various diseases and injuries that primarily or secondarily affect the brain, such as Alzheimer’s disease or stroke [1]. Most people with dementia live at home and are cared for by one or more family members, often referred to as the informal caregivers [2]. Caring for a family member with dementia puts one at risk for depression, anxiety, and other health problems [3]. The benefits of electronic health (eHealth) support for caregivers are increasingly recognized in dementia care practice because it offers an opportunity for accessible tailored interventions. Furthermore, eHealth interventions may fill the expected gap in supply and demand caused by demographic aging and the decreasing working population [2,4].

Previous research has found that specifically multicomponent, tailored eHealth interventions are promising for increasing caregiver confidence and self-efficacy and decreasing caregiver stress and depression [5,6]. Furthermore, blending online and face-to-face support may increase caregiver-therapist relations and adherence [7]. Next to intervention delivery, intervention timing might be crucial for efficacy. Early support may help people with dementia and their caregivers adapt to the changes of early dementia and feel more competent in their care skills. However, the current contrast between highly advanced timely diagnostic tools and the lack of available support to match this early diagnosis is troubling [8]. Early interventions can offer support in coping with an insecure future and preparing caregivers for the possibility of further decline and dependency [9] and have been proved to be effective in reducing strain and delaying institutionalization of the person with dementia [10,11]. However, some caregivers may feel stigmatized and refuse help if the information does not fit their personal situation and stage of the disease [12]. Existing programs are mostly aimed at problematic behaviors that occur in the advanced stages of dementia [13-17], or are not specifically adapted to the needs of caregivers during the early stages of dementia [18]. Early-stage interventions may prove to be beneficial if they focus on adaptation to the caregiver role and actively involve caregivers to tailor the intervention to their needs [9,19].

The Medical Research Council (MRC) framework suggests that intervention developers should put effort into the actual use of effective interventions by considering the implementation during the first phases of development and evaluation [20]. Following these recommendations, the blended care program “Partner in Balance” (PiB) for early-stage dementia caregivers was developed together with potential users. The program focuses on caregivers’ capacity to fulfill their potential and obligations and to help them manage their lives with some degree of independence and to engage in social activities together with the care recipient [21], fitting the new definition of health for chronic conditions [22]. To evaluate the program’s effectiveness, a randomized controlled trial (RCT) was performed in the context of daily care practice. Several caregiver eHealth support studies lack methodological rigor [6], and RCTs are considered proper designs for convincing evidence. However, before the effect analysis, evaluation of the sampling quality and intervention quality is essential for the interpretation and generalizability of results and further fine-tuning (or even annulling) of the effect analysis [23]. Furthermore, knowledge of implementation barriers and facilitators in an effect study can aid future adoption and implementation of a new intervention in care practice. Generalization and applicability of findings, for example, the public health significance of interventions (external validity), are particularly of interest for clinicians and policy makers. Implementation can be complicated in the complex and heterogeneous structure and culture of dementia care organizations [24]. Furthermore, the Consolidated Standards of Reporting Trial (CONSORT)-EHEALTH reporting standards for eHealth interventions place additional focus on external validity by assessing the following: (1) the context within participants benefitting from the intervention; (2) the delivery mode, features, and functionality of the intervention; (3) the use of prompts to interact with the intervention; and (4) any cointerventions that may occur. Additionally, a newly developed program can be effective but difficult to implement in care practice if health professionals or policy makers do not accept it. The implementation aspects (ie, costs and intervention description, including frequency, type, and duration of contacts) are important according to the CONSORT-EHEALTH standards. Therefore, process evaluations should focus on internal validity and implementation knowledge to increase credibility [23].

Different frameworks for process evaluations have been proposed, but there is no consensus on elements that process evaluations should cover [23]. In this study, we describe first-order (sampling and intervention quality) and second-order (implementation knowledge) process data based on the model of Leontjevas et al [23] to assess the internal and external validity of the PiB trial and its implementation to inform effect analysis. Similar to the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) model, this model included the recommendations fitting the CONSORT-EHEALTH standards defining the internal and external validity.

The Medical Ethics Committee of the Maastricht University Medical Center+ (MUMC+) approved this study (#12-4-059). The process evaluation was conducted alongside the effectiveness study. Detailed information on the methods are presented in a previous study [25]. A short description is provided below.

A randomized waiting-list controlled trial was conducted in the southern part of the Netherlands. Family caregivers were recruited within memory clinics, mental health care organizations, and caregiver support services. The intervention was delivered and evaluated within these organizations. A waiting-list controlled design was chosen to increase acceptability and adherence to the research protocol in the control group and decrease attrition effects [20]. Data were collected pre- and postintervention and at 3-, 6-, and 12-month follow-up points.

Detailed information about the program components and development is presented in a previous study [21]. In short, the blended care self-management program PiB consists of the following: (1) a face-to-face intake session with a personal coach to familiarize participants with the program, set goals with the goal attainment scaling (GAS) method, and select preferred module themes [26]; (2) tailored online thematic modules, including psychoeducation, behavioral modeling, reflective assignments, change plans, and email feedback from the coach over 8 weeks; and (3) a face-to-face evaluation session with the coach evaluating previously set goals. Furthermore, the participants can interact with other participants via the discussion forum. The participants are free to set their own personal goals. The module themes are acceptance, balance in activities, communication with family members and environment, coping with stress, focusing on the positive, insecurities and rumination, self-understanding, the changing family member, and social relations and support. The participants choose 4 modules; 2 weeks were allocated for each module. However, the participants were allowed to complete the modules at their own pace in accordance with the self-management approach [27]. The personal page and modules remained accessible for participants after the intervention period. The control group consisted of an 8-week waiting list while receiving usual care (nonfrequent counseling). After the posttest assessment, they were given the opportunity to follow PiB.

The personal coaches were trained, experienced professionals (psychologist or psychiatric nurses) from one of the participating organizations. They attended a 2-hour training session in self-management techniques, goal setting and online help, and regular supervision meetings. Their tasks were familiarizing participants with the online program, supporting them in module choice and goal setting, and giving feedback on the self-reflective assignments through the online messaging portal of the program, which was accessed via email.

First-order (sampling and intervention quality) and second-order process data (implementation knowledge) were collected following a process evaluation framework based on earlier research [23,24].

Descriptive statistics (SPSS Statistics V22.0, IBM USA, NY, 2013) were used for the quantitative data analysis. The interviews were audiotaped and transcribed verbatim. The deductive content analysis was used for the qualitative data analysis (open-ended questions and transcripts) by authors LB and CS using the qualitative analysis software ATLAS.ti (V1.0.4, GmbH Germany: Berlin, 2014). The first author developed an unconstrained analysis matrix based on the interview themes. Both authors (LB and CS) coded the data for correspondence with or exemplification of the identified categories. Data not fitting the unconstrained matrix served as the basis for additional categories created following the principles of inductive content analysis; conceptual labels were assigned to textual fragments and were organized into categories. Saturation occurred after coding 40 semistructured interviews (eg, no new categories emerged). Existing and newly developed categories were merged into common themes in a consensus meeting (LB and CS). Disagreements were solved through discussion, together with author MV. To determine goal attainment, raw GAS scores were transformed into an individual mean GAS score (T-score). The T-scores included the attainment level and a potential weight assigned to the goals. T-scores of ≥50 indicate effective goal achievement [26].

This study described the process of PiB to explore its credibility and generalizability. First-order and second-order process data were evaluated from the perspective of both family caregivers (participants) and professionals delivering the intervention (coaches) to increase the understanding of the mode of delivery [30].

The data on sampling quality showed that the participation rate of caregivers was 51.9%. Considering that the average response rate is 27% for caregiver research, our participation rate can be considered substantial [31]. However, these 51.9% only included eligible caregivers who were already familiar with the care parties involved in recruitment. Therefore, they may have been highly motivated and open to support [32]. Recruitment barriers were lack of computers, lack of need for intervention or additional support, etc, which were also reasons for the respondents to decline participation. During the early stages, caregivers may struggle with a fear of stigma and low acceptance [19]. This might explain why some participants struggled with accepting their own crucial role in any desired changes and why participants were not ready to openly discuss their issues in the discussion forum. However, the low use of the discussion forum could also be a consequence of the abundance of currently available online communication tools, eliminating the need for yet another form of online communication.

Young age and employment were considered recruitment facilitators. This finding is congruent with the findings of previous research stating that lower age correlated to higher eHealth literacy, that is, the skills and knowledge necessary to use online health resources [33,34]. However, seniors’ use of the Internet is expected to rise in the near future, increasing the accessibility of eHealth programs, such as PiB [35]. Furthermore, the results show that spouses and children of people with young onset dementia had difficulty identifying with the older caregivers in the examples. Previous research confirmed that younger caregivers struggle with different aspects in daily life compared with older caregivers [36]. It is essential to match program content to the specific needs of the target audience to maximize the benefit of a supportive intervention [6,37,38]. The thematic structure of the program allows for an add-on of specific themes for subgroups, such as caregivers of people with young-onset dementia.

At the postintervention assessment (n=68), the blinding of allocation was intact for only 46% (31/68) of the participants, which can potentially bias the estimation of effectiveness. However, blinding in psychosocial research can be challenging and it is rarely reported if blinding is maintained [39]. This item from the CONSORT may have been developed with studies of pharmacological treatments in mind, but blinding or masking may be unfeasible for certain aspects of psychosocial treatment studies [40]. Other important aspects of psychosocial RCTs need to be considered and reported but are not addressed in the CONSORT statement, for example, adequate reporting of treatment integrity (whether therapists were consistent in providing the specified intervention). Verification of treatment integrity, or fidelity, in outcome studies is essential to ensure that valid comparisons of replicable treatments can be made [41].

The professionals were satisfied with the intervention being manageable considering their busy schedules, giving them time to focus on their feedback. They reported a more profound relationship with the caregiver; the program allowed them to empathize with the caregiver. However, the professionals considered the nondirective attitude toward the participants a challenge in the self-management approach. This issue deserves further exploration because the performance of the self-management health care provider is essential for the participant’s performance of self-management tasks and overall intervention effectiveness [42]. It was previously argued that health care providers do not always support self-management education because this is not part of their attitudes, beliefs, or regular care practice [27]. To maximize the effects of self-management programs, increasing the essential clinical competences of health care providers to provide tailored self-management support [43] and raising awareness of the benefits of evidence-based self-management programs for their target population [27] are recommended. For example, this study showed that both participants and coaches mentioned a more profound relationship with one another, which was also demonstrated in a previous blended-care intervention for depression [44]. Previous research confirmed that the opportunity to reflect on one’s feelings anonymously in a private and safe environment is easier than doing so in person, but face-to-face contact increased caregiver openness and coach empathy [7]. The participant compliance with all 4 modules was high (87.8%), which could be explained by the motivational aspect of having a coach [7]. The varying intervention period and dose may influence the effectiveness of the intervention [41]. However, reasons for protocol deviations were diverse and not uncommon for informal caregivers and elderly participants (eg, caregiver pace, time constraints, and illness) [31]. In self-management interventions, the participant is in control and should therefore be allowed to complete the modules at his own pace [27,45]. Furthermore, complex interventions such as PiB are often designed to be implemented with some flexibility to accommodate differences among participants [46].

PiB was evaluated within multiple organizations with coaches from different backgrounds who received training and supervision in self-management and blended care. A relatively high response rate of organizations was found (73%), which could be attributed to the current demand of health care insurance for care organizations to provide eHealth [47]. However, a lack of financing and time could hamper the implementation. To overcome the experienced barriers and implement the program on a larger scale, awareness of the benefits of blended care self-management programs is required in addition to the training of self-management skills for the health care providers. The MRC Framework suggests that implementation should be considered during the first phases of intervention development and evaluation [48]. Involving stakeholders in technological development and evaluation can facilitate implementation in different care settings [49]. As complex interventions are influenced by context, several psychosocial interventions show different results in different settings or countries [40]. Additional information about the process of the implementation is crucial to understanding why an intervention is effective in one setting and not in another [46]. Before implementing the program on a larger (international) scale, barriers and facilitators for implementation should be identified with regard to possible differences based on organizational and cultural contexts. PiB was evaluated within different settings with coaches of different backgrounds delivering the intervention. To evaluate whether the background of the coach has any influence on the intervention outcome, this variable can be considered in the effect analysis. Additionally, the higher rate of dropout in the intervention group showed that this program can be considered burdensome. eHealth interventions in general are not appropriate for caregivers who are not computer literate or have more practical care needs. Several other factors influence the interaction between people with dementia and their caregivers, for example, caregivers’ personalities, psychological well-being, and the psychological symptoms of the care recipient. For instance, our results showed that this early-stage intervention came too late to help some caregivers. Caregivers were included based on the stage of the disease of their loved one, but some caregivers had been struggling with insecurities for years while the stage of the disease was still considered “early.” This highlights the need for tailored interventions, not only for the stage of the disease but also for the personal experiences, capacities, and other factors that may contribute to the intervention efficacy. Future research should consider including larger samples to examine the impact of eHealth interventions for subgroups of caregivers to tailor the care offered more efficiently [5]. However, in this study, a relatively high response and participation rate was found, indicating that having the option to choose this type of caregiver support is needed at a minimum. An active role of health care professionals in outlining care and support options early in the dementia process is recommended. Furthermore, sustainability of long-term intervention effects should be evaluated, and a cost-consequence or cost-effectiveness analysis should be conducted to inform decision makers of the value of PiB.

Several study limitations need to be considered. First, in our study protocol, deviations were measured with the self-report questionnaire for coaches. Previous research measuring protocol deviations with self-report questionnaires and ratings based on audio recordings found large discrepancies between the two measurements, indicating that professionals may not always be aware of their level of treatment fidelity [50]. Second, tracked usage data were measured in clicks. Clicks represent page views, but this clickstream method has a large disadvantage; several people who click on a page do not necessarily read it. Furthermore, 1 in 3 visitors spend less than 15 seconds reading the page, so a measured click does not automatically mean that the attention of the visitor is directed to the information on the page that is being viewed [19]. However, the results showed that the participants spent most of their time on the assignments and change plans and email contact with their coach, which represent the essential elements of a blended-care program [7,44]. Third, GAS was used to rate goal attainment. Goal setting and rating are considered a therapeutic task and were therefore performed by the coaches during the face-to-face sessions. Future research could consider setting and rating goals by an independent research assistant in all treatment arms to consider GAS as an outcome measure. However, GAS is a challenging evaluation method when not all participants set or evaluate goals, goals change during the process, or participants lack insight, communication skills, or the capacity to specify goals, and therefore should be used merely as a complementary scale [51].

The participants and professionals were satisfied with the intervention, but adapting the content to specific subgroups such as younger caregivers was recommended. Implementation of the program requires more awareness of the benefits of blended care self-management programs and training in tailored self-management skills for the health care provider. Overall, PiB can be considered a valuable addition to the existing caregiver support because it is tailored to the needs of the target audience and deepened the coach-caregiver relationship.