We conducted this systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline, 2020 [24]. Before the start of the study, the review protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO; CRD42022329669).

Two researchers (J-CH-R and CG-M) performed an independent electronic search in PubMed, Scopus, Web of Science, Cochrane Library, and CINAHL. The search included all records from the inception of the databases up to July 10, 2022. Search terms for digital strategies (“virtual,” “online,” “web-based,” “internet-based,” “digital,” “e-Health,” “m-Health,” “App,” and “mApp”), skin cancer (“melanoma,” “cutaneous melanoma,” “malignant melanoma,” and “skin cancer”), prevention (“prevention” and “sun protection”), and risk factors (“tanned,” “sunburn,” and “UV exposure”) were employed. These were combined using the Boolean operators “AND” and “OR.” Details of the search strategy can be found in Appendix S1 in Multimedia Appendix 1.

We developed the eligibility criteria following the PICOS model (ie, patient, intervention, comparison, outcome, and study design) shown in Table 1.

To manage data, Mendeley Desktop (version 1.19.8; Elsevier) was used to detect duplicates and carry out the screening process. Two independent researchers (J-CH-R and CG-M) screened records by title and abstract, and later performed a complete read of the studies to select those that met the mentioned criteria. Any disagreement was deliberated with a third researcher, J-JP-R.

We assessed methodological quality and risk of bias using The Cochrane Risk of Bias tool version 2 (ROB-2) [25]. This tool is composed of the following five domains: bias from randomization process, intended intervention, missing outcome data, measurement of outcomes, and selection of the reported results. The overall judgment is classified as “low,” “some concerns,” or “high” risk of bias. We also conducted subgroup analysis to determine how dropout events could be affected by the level of methodological quality and methodological threats such as blinding.

The following data were extracted from the RCTs included in the systematic review: authors or year and country, study population, recruited sample, analyzed sample, sex, experimental and control intervention, dropout rate, reasons for dropouts, and length of intervention. When the number or rate of dropouts was not directly provided in the manuscripts, both were calculated.

A dropout was considered when a participant did not complete the intervention or follow-up period, after the randomization process. For studies that included more than 2 groups of intervention, we separately analyzed the comparison groups two by two. Dropout data were extracted from the text of the randomized controlled trials provided in either a flowchart, in the description of participants, in the results sections, or in the discussion.

To analyze data, we used the free software R Studio version 4.1.1. (R foundation for Statistical Computing) metafor (version 3.0-2) [26], meta (version 5.1-1) [27], and dmetar (version 0.0.9000) [28] packages. The analysis consisted of overall and subgroups proportion and odds ratio (OR)–based meta-analyses and metaregression.

A random-effects model was employed in all meta-analyses considering possible heterogeneity between our selected RCTs. Furthermore, heterogeneity was assessed with I², with values exceeding 50% indicating large heterogeneity. The subgroup meta-analysis and metaregression was run when at least 3 arms of study were available.

The meta-analysis of proportions allowed us to calculate the overall pooled dropout rate with its 95% CI of all arms of the studies included in our review [29,30]. Additionally, a subgroup analysis was performed to calculate the pooled dropout rate for digital or comparator interventions and to determine which type of intervention resulted in the highest dropout rate. This analysis was complemented by an OR subgroup analysis ordered by digital or intervention comparator to determine whether the probability of losing the participants was greater in one group or another.

The OR meta-analysis evaluated whether the event (dropout) was more or less frequent in the digital or comparator intervention. When the OR was less than 1, dropouts were less likely in digital strategies. To assess the measure of effect on binary outcomes, the OR with a 95% CI was calculated, and the inverse variance method was used to adjust the pooled estimations to sparse data. The restricted maximum-likelihood estimator for τ² estimated the variance among RCTs [31]. When studies reported zero events in one or all groups of intervention, we added a 0.5 continuity correction to the meta-analyses so that these studies could contribute to the overall sample size of the review [32]. The OR meta-analyses were conducted and subsequently described in terms of absolute values. The results of the meta-analyses were displayed in forest plots.

A sensitivity analysis was carried out to detect how studies influenced the effect size. When a study was identified as an outlier based on the dropout variable, it was removed from the analysis. Furthermore, to confirm previous results, we performed an exploratory analysis using the L’Abbé, Baujat plot, Leave-One-Out meta-analysis, and influence plot.

A univariate metaregression analysis based on a random-effects model assessed the continuous variables of age, female percentage, male percentage, length of intervention in months, and sample size as covariates of the occurrence of dropouts. These predictors were selected to determine how the characteristics of the participants and interventions could influence dropouts [33]. Bubble plots were used to illustrate how a covariate modified the effect size in the metaregression analysis.

We examined the effects of small studies and publication bias based on the symmetry of the contour-enhanced funnel plot. The Harbord and Egger bias test were used to confirm the absence of asymmetry in the funnel plot (P>.05).

A total of 1566 studies were identified in the database search. After removing duplicates, the screening process, and complete reading of the records that met the eligibility criteria, 17 RCTs were finally included in the review [34-50]. The complete process is shown in Figure 1. Details of the excluded records are presented in Table S1 in Multimedia Appendix 1.

Regarding methodological quality, 14 (82%) of 17 RCTs showed “some concerns” based on the summary score of ROB-2. Moreover, 2 (12%) RCTs [44,47] showed a “low“ risk of bias, and only 1 (6%) had a “high” risk of bias [49] (Figure S1 in Multimedia Appendix 1). The latter showed a “high” risk of bias because baseline differences between groups were observed.

Regarding the subgroup analyses, an analysis sorted by participants’ blinding condition could not be performed because most of the studies were not blind or the blinding was not clearly specified. The subgroup meta-analysis sorted by the ROB-2 scores (Figure S2 in Multimedia Appendix 1) showed that a “low” overall score could indicate lower attrition in nondigital prevention strategies. However, due to the limited number of “low”-risk studies, the results should be interpreted with caution.

A sample of 6593 healthy participants and people free of disease during the study period was analyzed. The age of the participants ranged from 12.6 to 54.3 years. The digital strategies used in the included RCTs were web-based interventions in 8 studies [35,40,42,43,46,48-50], 6 involved apps [36-38,44,45,47], 3 involved SMS [35,36,39], 2 involved video [34,41], and 1 involved a wearable device [47]. Conversely, the comparator groups involved no intervention in 11 studies [35,37,43-48,50] and active controls in 6 studies [34,36,38,39,41,49].

The total number of dropouts for all arms of the included studies was 1120, with 681 (60.80%) in experimental interventions and 439 (39.20%) in controls. The reason for the dropout of participants was reported as loss during follow-up in 9 of the 17 RCTs [34,37,38,40-43,47,49] and not answering the final questionnaire in 4 studies [39,44,45,50]; 2 studies did not report the reason for dropout [35,48]. The main characteristics of the studies are displayed in Table 2.

The initial sensitivity analysis included a total of 23 arms from the randomized controlled trials of the review. After the sensitivity analysis, the study conducted by Brinker et al [44] was removed because it was identified as an outlier that influenced the effect size. The details of the sensitivity analysis are shown in Figures S3-S6 in Multimedia Appendix 1. Figure S7 in Multimedia Appendix 1 shows a funnel plot with absence of asymmetry, as confirmed by the Harbord test (P=.66) and Egger bias test (P=.69).

The meta-analysis of proportions included 22 arms (k) of study and 2610 subjects among whom there were 419 dropouts. An overall pooled dropout rate of 9.5% (95% CI, 5.0-17.5) was calculated (Figure 2; [34-36,38-50]). In the subgroup meta-analysis, digital strategies showed a higher dropout rate of 11.6% (95% CI 6.8-19.0) compared to 10.0% (95% CI 5.5-17.7) in the comparators. These results are displayed in forest plots, respectively, in Figures S8 and S9 in Multimedia Appendix 1.

A slight trend for a higher number of dropouts was observed in digital strategies with an OR of 1.16 (95% CI 0.98-1.36), but there were no significant differences between the experimental and control approaches (P=.39). The I² was 6% (95% CI 0-38) indicating a lack of heterogeneity between the studies analyzed for the overall and subgroup meta-analysis (Figure 3; [34-36,38-50]).

We performed a meta-analysis of subgroups divided by the type of digital strategy and the comparison groups. Only the strategies that were analyzed in more than two RCTs were included in the OR meta-analysis. As Figure S10 in Multimedia Appendix 1 shows, none of the digital interventions assessed differed significantly in the number of dropouts compared with the comparator strategies. The OR score was 0.88 for SMS (95% CI, 0.30-2.53), 1.17 for web-based interventions (95% CI 0.94-1.47), and 1.44 for Apps (95% CI 0.88-2.35). Our findings in the comparator subgroup analysis showed no significant differences, even when comparing digital strategies with active control (OR 1.13; 95% CI 0.82-1.54) or no-intervention groups (OR 1.14; 95% CI 0.90-1.44; Figure S11 in Multimedia Appendix 1).

Univariate metaregression analysis (Table 3) for age, female percentage, male percentage, and length of intervention in months and sample size did not show any significant association with the effect size of the study. Metaregression bubble plots for these analyses are presented in Figures S12-S16 in Multimedia Appendix 1.

This systematic review synthesizes information on the attrition of RCTs based on eHealth interventions for the prevention of skin cancer. Quantitative analysis evaluated the pooled dropout rate and dropout OR, in addition to moderators that could influence the dropout of subjects in the meta-analyzed RCTs. Although the digital strategies employed within studies used different platforms or devices, all of them were focused on skin cancer prevention and were supervised by expert dermatologists.

The meta-analysis of proportions showed a pooled dropout rate of 9.5%, with a dropout rate of 11.6% and 10.0% for the eHealth interventions and comparators, respectively. These results are in line with the findings by Walters et al [51], who reviewed the retention in RCTs of health technology programs in the United Kingdom. This review established that there was a dropout rate of up to 11% in a significant proportion of RCTs. Dropout rates of 5% are likely to introduce bias, while if 20% is exceeded, this could affect the validity of the study due to insufficiency during data analysis [52,53]. No background research was found performing similar analyses in the dermatology literature, so the comparison of rates was not viable.

Eysenbach et al [20] hypothesized that the nature of digital strategies tends to a higher loss of participants, a phenomenon called the “Law of attrition.” Although a slightly higher dropout rate was observed in digital strategies compared with comparator groups in our proportion and OR meta-analysis, the difference was not significant. Our findings refute the “Law of attrition” in those studies that aim to prevent skin cancer through these innovative interventions.

Previous systematic reviews, such as Bevens et al [54], focused on the analysis of attrition of digital strategies in people with multiple sclerosis and found no significant differences between dropout rates in participants allocated to digital or control interventions. Although our findings are in line with these previous findings, the target population and research conditions differed from ours, so comparison of findings are difficult.

As in the overall OR meta-analysis, the subgroup meta-analysis sorted by type of digital strategy and comparators found no significant differences in dropout rate. Only SMS text messaging presented a lower odd of dropout compared with other digital interventions, but without statistical significance. Reminder-based interventions such as SMS seem to promote adherence in chronic conditions, but further research is still needed [55]. It is noteworthy that the dropout rates in participants allocated to no intervention showed losses similar to digital strategies, reflecting the prior expectation that they could be affected by nonexperimental factors and the loss of perspective of therapeutics goals [56].

Our metaregression found that none of the covariates moderated the interventions’ effect size. Nonetheless, Torous et al [17] obtained higher dropout rates in studies with larger sample sizes that used apps for depressive symptoms, possibly related to a lower rate of individual follow-up and feedback from subjects.

In addition to the moderator analysis, assessment of the reasons for dropping out could be a way to identify barriers to reduce attrition in future RCTs. However, the lack of transparency and homogeneity in reporting reasons for participants’ dropout in the studies included in this review made the aforementioned task challenging. The main reported cause of attrition in our RCTs was loss to follow-up, but this aspect did not show the real reason for the loss of participants.

As previously mentioned, dropout could threaten internal or external validity in studies. We recommend that researchers use our overall pooled dropout rate to calculate the sample size of future trials, avoiding possible threats. The overrecruitment of 10.1% in the sample size of RCTs may be a suitable way to overcome external validity risks [57,58].

Although our OR meta-analysis showed no differences in attrition between digital strategies and comparator interventions, in order to obtain conclusive results that can be turned into daily clinical practice, we point out the need for further research with head-to-head comparison between digital and conventional interventions (eg, education programs or brochures) for the prevention of skin cancer [59]. Dropout rates have previously been directly related to the acceptability and feasibility of the intervention [60,61].

Given the scarce information and lack of transparency provided by studies when reporting the number of and reasons for dropouts, a deep change in the research framework is needed. To overcome this obstacle, relevant guidelines such as Consolidated Standards of Reporting Trials report the need to detail the reasons and the number of participants lost during the study period [62,63]. Accurately following these guidelines would pave the way for researchers to find suitable dropout prevention plans. Previous literature, based on user experience with digital strategies, indicates that reliability, lack of technological education, lack of satisfaction with intervention, and sparse human feedback seem to be the main barriers to their use [63-65]. We encourage future researchers who aim to develop a digital strategy or perform RCT protocols to implement solutions to the mentioned barriers such as gamification, tailored and customizable e-interventions, personalized feedback, or programmed reminders (eg, mail and SMS). The gamification principles of meaningful purpose, meaningful choice, supporting player archetypes, feedback, and visibility proposed by Floryan et al [66] could enhance the user experience and engagement within digital health interventions. Gamification could increase motivation, reinforce learning objectives, and increase enjoyment and positive experiences in dermatological education and prevention approaches [9]. Likewise, programmed reminders are an effective way to promote prevention habits, highlighted by the use of text messages in dermatology [64]. Reminders associated with professional supervision have shown even greater results in prevention programs [67].

Given that RCTs are the first step required to translate research results into clinical settings, success in decreasing the number of participants dropping out within the research context could improve long-term engagement in digital programs for the prevention of skin cancer.

This review has several strengths. Our study provided an initial analysis of the dropout from RCTs to prevent skin cancer through digital strategies. Our computed rates could help calculate sample sizes in future studies. We performed a sensitivity analysis that helped us detect outliers and confirm the absence of publication bias. Moreover, the subgroups and metaregression analyses allowed us to understand how loss of participants could be modified by different predictors.

The main limitation of our review is that potential literature from other databases with non-English records could have been missing. Furthermore, our outcomes may have been conditioned by the heterogeneity of the experimental interventions in the included studies. Some of the studies compared digital strategies with no intervention, so we cannot assert that dropouts from these groups could be related to external factors. Evidence from the subgroup meta-analysis sorted by an active comparator group should be interpreted with caution because of the low number of analyzed studies; further research is needed to obtain strong evidence. We were unable to propose tailored advice to improve retention for this kind of RCT owing to the sparse information on reasons for dropout provided by the authors.

This systematic review and meta-analysis calculated an overall pooled participant dropout rate of 9.5% (95% CI 5.0-17.5), which should be considered in the calculation of sample size in RCTs aimed at preventing skin cancer using digital health interventions. Although a slightly higher pooled dropout rate was recorded for digital strategies, the OR-based meta-analysis did not show significant differences against the comparator groups. Our meta-analyses of subgroups sorted by digital and comparator interventions did not present significant statistical differences. Age, sex, length of the intervention, and sample size did not modify the effect size, so they were not moderators of dropout. We highlight the need to follow the guidelines and standardize reporting of the number of and reasons for participants’ dropout because this will be the only effective way to design a successful plan to reduce the loss of participants in studies that analyze digital approaches to prevent skin cancer.