The population of interest included informal caregivers aged 18 years and older who were currently providing caregiving support to adults aged 18 years and older (ie, care recipients) living in the community with at least 1 chronic condition.

Studies selected for this systematic review included those that examined any internet-based modality to deliver an intervention, which could include either a single component program or multimodal program to informal caregivers. An internet-based program was defined as any Web-based series of instructions, options, plans, lessons, modules, or curricula.

The primary outcome of interest for this systematic review was mental health, specifically including depressive symptoms, stress/distress, anxiety, coping, overall mental health, quality of life, and overall health. A second paper on other caregiver outcomes reported in these studies (eg, self-efficacy, self-esteem, burden) is in progress.

Full data extraction, including characteristics of included studies, was completed by 1 reviewer and verified by a second reviewer. Risk of bias found in individual studies was assessed by 1 reviewer and verified by a second reviewer. Risk of bias was assessed using the Cochrane risk of bias framework [26], which evaluates the level of bias for sequence generation, allocation concealment, blinding, completeness of outcome assessment, selective reporting, and other biases. The quality of the clinical evidence was critically appraised by 1 reviewer and verified by a second reviewer using the Grading of Recommendations Assessment, Development, and Evaluation system (GRADE), which evaluates the risk for bias, inconsistency, indirectness, and imprecision for each outcome [27]. Disagreements were resolved through consensus between the 2 reviewers.

A meta-analysis was used to combine the results across studies by outcome using the published data from included studies. To perform the meta-analysis, we used immediate posttreatment data (mean, SD) for continuous outcomes such as depression, stress or distress, anxiety, coping, overall mental health, quality of life, and overall health. We used intention-to-treat outcome data where possible; however, if no intention-to-treat data were reported, we used study completer’s outcome data.

The DerSimonian and Laird random effects models with inverse variance method were used to generate the summary measures of effect in the form of standardized mean difference (SMD) [28]. The SMD accounts for similar outcomes measured using different assessment tools (eg, depressive symptoms were assessed using different outcome measures such as Center for Epidemiologic Studies Depression Scale and Beck Depression Inventory) [29]. In this situation, it was necessary to standardize the results of the studies to a uniform scale before they could be combined in a quantitative synthesis. SMDs were calculated using change from baseline data for intervention and control groups for each study with relevant outcome data. For each outcome, data from the corresponding study were used to calculate the mean difference between pretreatment (baseline) and posttreatment (final or end point) values along with its SD for both intervention and control groups. In studies where the SD was not reported, we calculated the SD from the reported standard error (SE) of the mean, 95% confidence intervals (CIs) and P values or z scores using equations provided in Chapter 7 and Chapter 9 of the Cochrane Handbook for Systematic Reviews of Interventions [30,31]. The SMD is interpreted based on its magnitude according to Cohen d recommended thresholds (~0.2=small effect, ~0.5=medium effect, ~0.8=large effect) [32].

The primary meta-analysis was to examine any type of internet-based intervention program by mental health outcome. The secondary meta-analysis was to examine the effects of specific types of internet-based intervention programs on mental health outcomes. Based on our previous work [24], intervention types were categorized accordingly: (1) internet-based information or education only, (2) internet-based information or education plus peer psychosocial support (PPS), (3) internet-based information or education plus professional psychosocial support (PFPS), (4) internet-based information or education plus combined peer and professional psychosocial support, and (5) internet-based intervention with telephone monitoring along with combined peer and professional psychosocial support.

Statistical heterogeneity of combined studies was examined using standard methods. The I² statistic was used to quantify the magnitude of statistical heterogeneity between studies, where I² of 30% to 60% represents moderate and I² of >60% represents substantial heterogeneity [33]. A P value of <.10 was used as a guide to indicate where statistically significant heterogeneity may exist, upon which a closer examination of study differences was performed. All analyses were performed using the software packages Review Manager (RevMan version 5.3; The Cochrane Collaboration), STATA version 14 (StataCorp LLC), and GRADEpro Guideline Development Tool.

The search resulted in 7923 unique citations that were screened independently by 2 project staff. At title and abstract screening, we excluded 7633 studies, leaving 290 studies to be screened at full-text. Of those 290 studies, we identified 13 studies (14 papers) that met the inclusion criteria for this systematic review. References lists of the on-topic systematic reviews and included studies were searched but no additional studies were added (Figure 1).

The purpose, methods, participants, intervention, and risk of bias details of the included studies are shown in Multimedia Appendix 2. From among the 13 included studies, there were 11 studies that were RCTs [34-45], 1 study that was a CCT [46], and 1 study that combined both RCT and CCT designs [47]. Five of the included RCTs were conducted in Europe [34-38], and 5 RCTs were conducted in the United States [41-43,45], of which 1 RCT reported relevant outcomes across 2 papers [39,40]. There was 1 RCT conducted in Canada [44]. The CCT was conducted across the United States, Puerto Rico, and Mexico [46], and the combined CCT and RCT was conducted across 3 European countries [47].

In regard to the type of chronic conditions among care recipients, 9 studies included patients with some form of dementia [34-38,41,45-47]. Cardiovascular health disorders were represented in 3 studies, of which 2 studies included only stroke survivors [42,43] and the other study included a mixed stroke population of stroke-related dementia combined with patients having Alzheimer disease and Parkinson disease [44]. One study was based on non–small cell lung cancer care recipients [39,40]. All included studies were considered small in sample size (≤150 subjects per arm) and had a short length of study follow-up (<6 months). One study included a slightly longer study follow-up time period of 1-year [43]. A majority of studies included informal caregivers aged older than 50 years (range 53.8 to 67.8 years) [35-44,47], except in 1 study that included family caregivers who were also partially working and therefore reported a slightly lower age [45]. Two studies did not provide information on the average age of caregivers [34,46]. More than half of the caregivers were female in all of the included studies (range 56.3% to 100%).

From among the 13 included studies, there were 2 studies (15%) that were categorized as having used an internet-based information or education only intervention [41,45], 3 studies (23%) having used an internet-based information or education plus PPS intervention [34,36,37], 1 study (8%) having used an internet-based information or education plus PFPS intervention [35], 6 studies (46%) having used an internet-based information or education plus combined peer and professional psychosocial support intervention [38-40,42-44,46], and 1 study (8%) having used an internet-based intervention with telephone monitoring along with combined peer and professional psychosocial support [47].

Studies had a comparison group defined as receiving no internet-based intervention that could have included minimal guidance on information resources or website use [34,37,38,41,42,44], usual care with or without additional information [36,39,40,43,45,47], printed information [46], or electronic communications (eg, e-bulletins) [35].

Outcome assessment tools used for relevant mental health outcomes varied across studies and are summarized in Multimedia Appendix 3. Among the 13 included studies, outcomes examined included depression (8/13), stress or distress (6/13), anxiety (2/13), coping (2/13), overall mental health (1/13), quality of life (4/13), and overall health (2/13).

The results of the critical appraisal of individual studies for level of bias for sequence generation, allocation concealment, blinding, completeness of outcome assessment, selective reporting, and other biases are shown in Figure 2. Overall, the Cochrane Risk of Bias (RoB) showed mixed quality of study methodology: 2 studies with low RoB [35,37], 3 studies with high RoB [36,40,47], and 8 studies with unclear RoB [34,38,41-46].

The meta-analysis included an examination of the impact of all internet-based interventions combined as well as an analysis of the impact of each type of internet-based intervention according to mental health outcome. All forest plots are shown in Multimedia Appendix 4.

Our systematic review and meta-analysis showed small to moderate beneficial effects of internet-based interventions on caregiver mental health including a reduction in symptoms of depression, stress or distress, and anxiety. The types of internet-based interventions that appeared to have a beneficial effect on mental health included information or education only on decreasing depression, stress or distress, and anxiety and information or education plus PFPS on reducing depression and anxiety. Critical appraisal determined a wide range of the quality of evidence but included a moderate quality of evidence for a modest effect size for a beneficial effect among the 2 specific types of internet-based interventions of information or education only and information or education and PFPS. Additional benefits were shown for the internet-based intervention of information or education plus combined peer and psychological support when it came to overall health among caregivers; however, this was based on a small sample size (<20) and a very low quality of evidence.

Accounting for the type of internet-based intervention revealed additional trends not shown when all types of internet-based interventions were combined. The results showed an approximate 20% increase in the magnitude of effect for stress or distress and an information or education only internet-based intervention among 2 studies, compared to when all 6 studies on stress or distress were combined. Symptoms of depression were improved for an information or education only internet-based intervention as well as for an information or education plus PFPS internet-based intervention, not shown when all 8 studies on depression were combined.

Based on the detailed critical appraisal and quality assessment of included studies, there are a number of possible reasons that consistent findings across the mental health outcomes were not shown. According to the GRADE scores, the quality of evidence was poor for a number of the outcomes examined, and none of the outcomes was rated as having high-quality evidence. This may reflect, in part, this new and evolving area of focus and the resulting lack of consistency across studies—for example, not all studies examined the mental health outcomes of interest, there was variability in the measurement tools used to assess the different mental health outcomes, the care recipients across studies differed, and too few studies examined the different types of internet-based interventions resulting in small numbers of studies for some outcomes. No differences were noted for multicomponent internet-based interventions on coping and overall mental health since these outcomes were only examined in a few studies. No differences were noted for quality of life perhaps due to small sample sizes and differences in types of interventions. Studies included in the subgroup analyses by type of internet-based intervention were judged to be predominately of low to very low quality of evidence suggesting a number of methodological limitations. Four studies had high risk of bias in the area of incomplete outcome data; 3 studies had high risk of bias for blinding of outcome assessment; and 2 studies had high risk of bias for sequence generation, allocation concealment, and blinding of participants or providers.

There were also many areas where risk of bias could not be assessed due to lack of information in the published papers. For example, risk of bias related to allocation concealment was rated as unclear in 11 of the 13 interventions assessed. Risk of bias related to blinding of participants and providers was rated as unclear in 9 of the 13 interventions examined. Risk of bias related to blinding of outcome assessors was rated as unclear in 7 of the interventions examined. The provision of more detailed information about trial procedures using the Consolidated Standards of Reporting Trials (CONSORT) guidelines for nonpharmacological interventions [48] would enable more accurate assessments of studies for bias and may over time help to elevate the quality of evidence in this area.

We examined the best studies (those with low risk of bias) [35,37,42] to see if there were further insights to be gained. These studies all demonstrated beneficial effects on mental health outcomes: depression [35,42], anxiety [35], stress [37], and quality of life [37]. However, they included quite different types of internet-based interventions. Blom’s [35] information or education plus PFPS intervention, targeted at caregivers of persons with dementia, included both a Web-based 8-week course and coaching, monitoring, and evaluation provided by a psychologist. Hattink’s [37] information or education plus PPS intervention, also targeted at caregivers of persons with dementia, included a personalized training portal and 2 to 4 months of course materials, interactive exercises, and connection with a Facebook community. Smith’s [42] information or education plus combined peer and professional psychosocial support intervention, targeted at spousal caregivers of stroke survivors, involved an 11-week educational program supported by an experienced cardiovascular nurse manager. These interventions had a lengthier intervention period than some of the other studies, 2 of these studies involved professional support, and 1 involved connections with other caregivers. It is possible that these intervention components hold more promise for improving mental health outcomes of caregivers.

Despite significant findings across a range of evidence quality, the intervention mechanism by which improvements in mental health were achieved is still not clear. The interactivity of the information or education only internet-based interventions may have contributed to our significant findings as previously shown by Guay et al [20]. The previously shown important role of human support [20] was variable in our findings, with a beneficial effect shown for the addition of professional psychological support only. It may be that the needs and experiences of the caregivers targeted in these multicomponent interventions are so diverse that the potential impacts of internet-based interventions are not realized. A theoretical basis for internet-based interventions [20,49] has shown to be impactful, and a number of our included studies reported using theory to develop their interventions [36,42,47]. Many interventions included behavior change techniques such as stress management [34,41], problem solving [35], and graded tasks [37], which may have contributed to significant findings. The most efficacious interventions included caregivers and care recipients who were homogeneous, with caregivers characterized as being mostly older female adults and care recipients being those living with some form of dementia [35-38,41,44,45]. Internet-based interventions, when designed with the target populations in mind, may be more likely to demonstrate a beneficial effect on the mental health of caregivers. Internet-based interventions being developed for caregivers should have a strong theoretical basis [50] and incorporate behavior change techniques, particularly those aimed to help manage stress and enhance coping.

This review summarizes the most relevant trial evidence available to assess the benefits of internet-based interventions on caregiver mental health outcomes. All of the available evidence was published between 2005 and 2017, with more literature published recently (from 2013 to 2017), emphasizing the growing interest in internet technology to support caregivers. However, the review identified that the overall quality of evidence ranged from very low to moderate quality. To our knowledge, this is the first systematic review and meta-analysis examining the impact of internet-based interventions on mental health outcomes of caregivers of adults with chronic conditions living in the community. Although this is an emerging field in the literature, our review set out an a priori selection of rigorous methodological designs, including RCTs and CCTs. This systematic review and meta-analysis was completed with a comprehensive search strategy developed to identify relevant and on-topic literature pertaining to internet-based interventions on informal caregiver mental health outcomes. The review was conducted using methodologically rigorous processes for systematic reviews and meta-analyzed the data using appropriate methods for combining studies that used different outcome assessment tools.

The limitations of the review include the methodological weakness of the studies included, despite being RCTs and CCTs. There was considerable heterogeneity in the interventions across studies. Therefore, we analyzed the impact of the internet-based interventions according to the components of the interventions to understand the impact of these components; however, there were too few studies having used each type of internet-based interventions across all of the mental health outcomes of interest.

This is the first meta-analysis of the impact of internet-based interventions for informal caregivers of adults with chronic conditions on caregiver mental health outcomes. The findings suggest there is an emergence of literature pertaining to internet-based interventions for informal caregivers examining the impact on mental health outcomes. However, future large, high-quality research with clear methodology and consistently reported outcomes of mental health using standardized assessment tools to facilitate meta-analysis and an assessment of clinical relevance are needed to further inform the effectiveness of such interventions, particularly multicomponent internet-based interventions that use peer or professional health care provider support.