This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Digital strategies are innovative approaches to the prevention of skin cancer, but the attrition following this kind of intervention needs to be analyzed.
The aim of this paper is to assess the dropouts from studies focused on digital strategies for the prevention of skin cancer.
We conducted this systematic review with meta-analyses and metaregression according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statements. Search terms for skin cancer, digital strategies, and prevention were combined to search PubMed, Scopus, Web of Science, CINAHL, and Cochrane Library from inception until July 2022. Randomized clinical trials that reported dropouts of participants and compared digital strategies with other interventions to prevent skin cancer in healthy or disease-free participants were included. Two independent reviewers extracted data for analysis. The Revised Cochrane Collaboration Bias tool was employed. We calculated the pooled dropout rate of participants through a meta-analysis of proportions and examined whether dropout was more or less frequent in digital interventions against comparators via an odds ratio (OR) meta-analysis. Data were pooled using a random-effects model. Subgroup meta-analyses were conducted in a meta-analysis of proportions and OR meta-analysis to assess the dropout events when data were sorted by digital interventions or control comparator. A univariate metaregression based on a random-effects model assessed possible moderators of dropout. Participants’ dropout rates as pooled proportions were calculated for all groups combined, and the digital and comparator groups separately. OR>1 indicated higher dropouts for digital-based interventions. Metaregressions were performed for age, sex, length of intervention, and sample size.
A total of 17 studies were included. The overall pooled dropout rate was 9.5% (95% CI 5.0-17.5). The subgroup meta-analysis of proportions revealed a dropout rate of 11.6% for digital strategies (95% CI 6.8-19.0) and 10.0% for comparators (95% CI 5.5-17.7). A trend of higher dropout rates for digital strategies was observed in the overall (OR 1.16, 95% CI 0.98-1.36) and subgroup OR meta-analysis, but no significant differences were found between the groups. None of the covariates moderated the effect size in the univariate metaregression.
Digital strategies had a higher dropout rate compared to other prevention interventions, but the difference was not significant. Standardization is needed regarding reporting the number of and reasons for dropouts.
International Prospective Register of Systematic Reviews (PROSPERO) CRD42022329669; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=329669
Digital strategies have experienced a boom in use in prevention programs for skin cancer in recent years. Primary and secondary prevention programs are the mainstay to reduce the incidence rate of skin cancer [
Digital strategies seem to be more effective in the prevention of skin cancer than other conventional strategies [
The engagement of the patients with the prevention and digital strategies determines their effectiveness. Despite the increasing interest of researchers in implementing RCTs that analyze digital strategies, there is still no consensus in the literature on whether they positively or negatively influence the dropout and adherence of participants [
Dropout or attrition is a constant challenge for researchers in RCTs and other longitudinal studies [
No previous studies have analyzed dropout in digital strategies for skin cancer prevention; therefore, our aim was to systematically assess and meta-analyze the existing RCTs to calculate the overall pooled dropout rate and to examine possible factors that could influence the dropout of users.
We conducted this systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline, 2020 [
Two researchers (J-CH-R and CG-M) performed an independent electronic search in PubMed, Scopus, Web of Science, Cochrane Library, and CINAHL. The search included all records from the inception of the databases up to July 10, 2022. Search terms for digital strategies (
We developed the eligibility criteria following the PICOS model (ie, patient, intervention, comparison, outcome, and study design) shown in
Eligibility criteria based on the PICOSa model.
PICOS model | Inclusion criteria | Exclusion criteria |
Population | Participants free of skin cancer during the study period | Participants with skin diseases during the study period |
Intervention | Digital prevention strategies | Preventions approaches not focused on digital strategies |
Comparator | Any type of comparator | Digital prevention strategies as comparator |
Outcomes | Number of participants who dropout during the study period | Studies in which the dropout number was not reported, or indirect calculation was not allowed |
Study design | Randomized controlled trials written in English | Any other type of study design |
aPICOS: patient, intervention, comparison, outcome, and study design.
To manage data, Mendeley Desktop (version 1.19.8; Elsevier) was used to detect duplicates and carry out the screening process. Two independent researchers (J-CH-R and CG-M) screened records by title and abstract, and later performed a complete read of the studies to select those that met the mentioned criteria. Any disagreement was deliberated with a third researcher, J-JP-R.
We assessed methodological quality and risk of bias using The Cochrane Risk of Bias tool version 2 (ROB-2) [
The following data were extracted from the RCTs included in the systematic review: authors or year and country, study population, recruited sample, analyzed sample, sex, experimental and control intervention, dropout rate, reasons for dropouts, and length of intervention. When the number or rate of dropouts was not directly provided in the manuscripts, both were calculated.
A dropout was considered when a participant did not complete the intervention or follow-up period, after the randomization process. For studies that included more than 2 groups of intervention, we separately analyzed the comparison groups two by two. Dropout data were extracted from the text of the randomized controlled trials provided in either a flowchart, in the description of participants, in the results sections, or in the discussion.
To analyze data, we used the free software R Studio version 4.1.1. (R foundation for Statistical Computing) metafor (version 3.0-2) [
A random-effects model was employed in all meta-analyses considering possible heterogeneity between our selected RCTs. Furthermore, heterogeneity was assessed with I2, with values exceeding 50% indicating large heterogeneity. The subgroup meta-analysis and metaregression was run when at least 3 arms of study were available.
The meta-analysis of proportions allowed us to calculate the overall pooled dropout rate with its 95% CI of all arms of the studies included in our review [
The OR meta-analysis evaluated whether the event (dropout) was more or less frequent in the digital or comparator intervention. When the OR was less than 1, dropouts were less likely in digital strategies. To assess the measure of effect on binary outcomes, the OR with a 95% CI was calculated, and the inverse variance method was used to adjust the pooled estimations to sparse data. The restricted maximum-likelihood estimator for τ2 estimated the variance among RCTs [
A sensitivity analysis was carried out to detect how studies influenced the effect size. When a study was identified as an outlier based on the dropout variable, it was removed from the analysis. Furthermore, to confirm previous results, we performed an exploratory analysis using the L’Abbé, Baujat plot, Leave-One-Out meta-analysis, and influence plot.
A univariate metaregression analysis based on a random-effects model assessed the continuous variables of age, female percentage, male percentage, length of intervention in months, and sample size as covariates of the occurrence of dropouts. These predictors were selected to determine how the characteristics of the participants and interventions could influence dropouts [
We examined the effects of small studies and publication bias based on the symmetry of the contour-enhanced funnel plot. The Harbord and Egger bias test were used to confirm the absence of asymmetry in the funnel plot (
A total of 1566 studies were identified in the database search. After removing duplicates, the screening process, and complete reading of the records that met the eligibility criteria, 17 RCTs were finally included in the review [
Regarding methodological quality, 14 (82%) of 17 RCTs showed “some concerns” based on the summary score of ROB-2. Moreover, 2 (12%) RCTs [
Regarding the subgroup analyses, an analysis sorted by participants’ blinding condition could not be performed because most of the studies were not blind or the blinding was not clearly specified. The subgroup meta-analysis sorted by the ROB-2 scores (Figure S2 in
Flow diagram of trials selection based on PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines.
A sample of 6593 healthy participants and people free of disease during the study period was analyzed. The age of the participants ranged from 12.6 to 54.3 years. The digital strategies used in the included RCTs were web-based interventions in 8 studies [
The total number of dropouts for all arms of the included studies was 1120, with 681 (60.80%) in experimental interventions and 439 (39.20%) in controls. The reason for the dropout of participants was reported as loss during follow-up in 9 of the 17 RCTs [
Summary of the included studies in the systematic review.
Source | Population | Recruited or analyzed (n) | Percentage of sex, age (years), or mean (SD) | Experimental intervention | Comparator intervention | Dropout rate (%) | Reason for dropouts (EG/CGa) | Length of intervention (months) |
Armstrong et al, 2011 [ |
English speakers aged >18 years | EG: 47/43; CG: 47/40; n=94 | Female: 50%; male: 50%; 37.2 years | Online video addressing how sunscreen works to protect skin | Active (brochure) | EG: 8.5% (4/47); CG: 14.9% (7/47) | Lost to follow-up | 3 |
Böttcher et al, 2019 [ |
Young organ transplant recipients | EG1: 44/39; EG2: 49/40; CG: 44/33; n=137 | Female: 44.5%; male: 55.5%; 12.6 years | EG1: SMS text message providing sun protection advice; EG2: WBIb with sun protection training | No intervention (waitlist) | EG1: 11.4% (5/44); EG2: 18.4% (9/49); CG: 25.0% (11/44) | N/Rc | 12 |
Bowen et al, 2019 [ |
First-degree relatives of melanoma cases | EG: 157/141; CG: 156/137; n=313 | Female: 63.6%; male: 36.4%; 51.3 years | WBI with weekly messages of melanoma prevention behaviors | No intervention (waitlist) | EG: 10.2% (16/157); CG: 12.2% (19/156) | Lost to follow-up | 12 |
Brinker et al, 2020 [ |
Secondary school pupils | EG: 734/734; CG: 839/839; n=1573 | Female: 51.6%; male: 48.4%; 15.9 (SD 1.3) years | App that modifies a selfie according to different levels of UV exposure for future 5 to 25 years based on individual skin type | No intervention | EG: 17.3% (127/734); CG: 6.20% (52/839) | Lost to follow-up | 6 |
Buller et al, 2015 [ |
Adults aged >18 years owning a smartphone | EG: 96/89; CG: 106/104; n=202 | Female: 73.5%; male: 26.5%; 33.3 (SD 9.8) years | App giving feedback on sun protection and alerted users to apply or to reapply sunscreen and to get out of the sun | No intervention | EG: 7.3% (7/96); CG: 1.9% (2/106) | Lost to follow-up and survey not completed | 3 |
Craciun et al, 2011 [ |
Female volunteers | EG1: 74/74; EG2: 70/70; CG: 61/61; n=205 | Male: 0%; female: 100%; 25.1 (SD 8.7) years | EG1: WBI volitional theory–based; EG2: WBI motivational theory–based | No intervention | 0% | Not applied | 1 |
Hacker et al, 2018 [ |
Young adults aged 18-35 years | EG1: 41/35; EG2: 42/36; CG: 41/36; n=124 | Female: 65.8%; male: 31.5%; 25.8 years | EG1: app that displays the daily UV index and gives sun protection advice; EG2: wearable with UV dosimeter | No intervention | EG1: 14.6% (6/41); EG2: 14.3% (6/42); CG: 12.2% (5/41) | Lost to follow-up | 3 |
Heckman et al, 2016 [ |
Adults aged 18-25 years | EG1: 287/195; EG2: 338/205; CG: 340/229; n=965 | Female: 66.1%; male: 33.9%; 21.8 (SD 2.2) years | EG1: WBI with a tailored intervention based on the Integrative Model of Behavioral Prediction; EG2: WBI with the Skin Cancer Foundation website | No intervention | EG1: 32.1% (92/287); EG2: 39.4% (133/338); CG: 32.7% (111/340) | N/R | 3 |
Hillhouse at al, 2017 [ |
Female adolescents | EG: 214/182; CG: 229/206; n=443 | Female: 100%; male: 0%; 15.2 (SD 2.0) years | WBI to reduce ITd motivations | Active (placebo) | EG: 15.9% (32/214); CG: 10.1% (23/229) | Lost to follow-up | 6 |
Manne et al, 2021 [ |
Participants at increased risk for melanoma aged 18-89 years | EG: 56/43; CG: 60/56; n=116 | Female: 69.8%; male: 30.2%; 51.1 (SD 15.2) years | WBI to improve SSEe and sun protection | No intervention | EG: 76.8% (13/56); CG: 93.3% (4/60) | Survey not completed | 3 |
Marek et al, 2018 [ |
Adults aged ≥18 years | EG1: 18/18; EG2: 17/17; EG3: 17/17; CG: 17/17; n=69 | Female: 61.1%; male: 38.9%; 54.3 (SD 13.9) years | EG1: app allowing total body photography; EG2: SMS to remind SSE; EG3: SMS+ accountability partner | Active (accountability partner) | 0% | Not applied | 6 |
Reilly et al, 2021 [ |
Adults aged >18 years who survived stage 0-2C primary cutaneous melanoma | EG: 121/82; CG: 119/86; n=240 | N/Af | App to encourage and improve SSE | No intervention | EG: 32.2% (39/121); CG: 27.7% (33/119) | Lost to follow-up | 12 |
Robinson et al, 2016 [ |
Kidney transplant recipients | EG: 84/78; CG: 86/83; n=170 | Female 40.6%; male: 59.4%; 50.0 years | App with educational sun protection content | Active (usual education) | EG: 7.1% (6/84); CG: 3.5% (3/86) | Lost to follow-up | 1.5 |
Robinson et al, 2021 [ |
Female adults | EG: 494/390; CG: 495/414; n=989 | Female: 100%; male: 0%; 47.0 years | SMS to remind SSE | Active (brochure) | EG: 21.1% (104/494); CG: 16.4% (81/495) | Survey not completed and discontinued intervention (EG) | 3 |
Stapleton et al, 2015 [ |
Female adults aged 18-25 years with IT in the past 12 months | EG: 94/74; CG: 93/85; n=186 | Female: 100%; male: 0%; 19.8 (SD1.4) years | WBI with psychoeducational content to reduce IT | No intervention | EG: 8.5% (8/94); CG: 8.6% (8/93) | No response | 1.5 |
Tsai et al, 2017 [ |
Adults aged ≥18 years | EG: 71/42; CG: 72/34; n=143 | Female: 74.1%; male: 25.9%; 42.3 years | Online melanoma video tutorial + brochure | Active (brochure) | EG: 40.8% (29/71); CG: 52.8% (38/72) | Lost to follow-up | 1 |
Vuong et al, 2018 [ |
General practice patients | EG: 134/89; CG: 138/96; n=272 | Female: 71.7%; male: 28.3%; 45.5 years | WBI with tailored melanoma risk assessment and prevention + usual education | Active (usual education) | EG: 33.9% (45/134); CG: 30.4% (42/138) | Lost to follow-up | 1.5 |
aCG: comparator group; EG: experimental group.
bWBI: web-based intervention.
cN/R: not reported.
dIT: indoor tanning.
eSSE: skin self-examination.
fN/A: not applicable.
The initial sensitivity analysis included a total of 23 arms from the randomized controlled trials of the review. After the sensitivity analysis, the study conducted by Brinker et al [
The meta-analysis of proportions included 22 arms (
Forest plot of overall meta-analysis of proportions for all groups of studies.
A slight trend for a higher number of dropouts was observed in digital strategies with an OR of 1.16 (95% CI 0.98-1.36), but there were no significant differences between the experimental and control approaches (
Forest plot of overall odds ratio meta-analysis for all groups of studies.
We performed a meta-analysis of subgroups divided by the type of digital strategy and the comparison groups. Only the strategies that were analyzed in more than two RCTs were included in the OR meta-analysis. As Figure S10 in
Univariate metaregression analysis (
Univariate metaregression analysis.
Covariate | Coefficient (95% CI)a | SE | ||
Age | 0.05 (–0.01 to 0.02) | 0.24 | –0.08 | .53 |
Percentage of female | 0.008 (–0.001 to 0.018) | 0.004 | 1.78 | .09 |
Percentage of male | –0.008 (–0.02 to 0.001) | 0.005 | –1.79 | .09 |
Length of intervention (months) | –0.023 (–0.07 to 0.03) | 0.023 | –0.98 | .34 |
Sample size | 0.0004 (–0.0002 to 0.0009) | 0.0002 | 1.45 | .16 |
aAccording to the random-effects model.
This systematic review synthesizes information on the attrition of RCTs based on eHealth interventions for the prevention of skin cancer. Quantitative analysis evaluated the pooled dropout rate and dropout OR, in addition to moderators that could influence the dropout of subjects in the meta-analyzed RCTs. Although the digital strategies employed within studies used different platforms or devices, all of them were focused on skin cancer prevention and were supervised by expert dermatologists.
The meta-analysis of proportions showed a pooled dropout rate of 9.5%, with a dropout rate of 11.6% and 10.0% for the eHealth interventions and comparators, respectively. These results are in line with the findings by Walters et al [
Eysenbach et al [
Previous systematic reviews, such as Bevens et al [
As in the overall OR meta-analysis, the subgroup meta-analysis sorted by type of digital strategy and comparators found no significant differences in dropout rate. Only SMS text messaging presented a lower odd of dropout compared with other digital interventions, but without statistical significance. Reminder-based interventions such as SMS seem to promote adherence in chronic conditions, but further research is still needed [
Our metaregression found that none of the covariates moderated the interventions’ effect size. Nonetheless, Torous et al [
In addition to the moderator analysis, assessment of the reasons for dropping out could be a way to identify barriers to reduce attrition in future RCTs. However, the lack of transparency and homogeneity in reporting reasons for participants’ dropout in the studies included in this review made the aforementioned task challenging. The main reported cause of attrition in our RCTs was loss to follow-up, but this aspect did not show the real reason for the loss of participants.
As previously mentioned, dropout could threaten internal or external validity in studies. We recommend that researchers use our overall pooled dropout rate to calculate the sample size of future trials, avoiding possible threats. The overrecruitment of 10.1% in the sample size of RCTs may be a suitable way to overcome external validity risks [
Although our OR meta-analysis showed no differences in attrition between digital strategies and comparator interventions, in order to obtain conclusive results that can be turned into daily clinical practice, we point out the need for further research with head-to-head comparison between digital and conventional interventions (eg, education programs or brochures) for the prevention of skin cancer [
Given the scarce information and lack of transparency provided by studies when reporting the number of and reasons for dropouts, a deep change in the research framework is needed. To overcome this obstacle, relevant guidelines such as Consolidated Standards of Reporting Trials report the need to detail the reasons and the number of participants lost during the study period [
Given that RCTs are the first step required to translate research results into clinical settings, success in decreasing the number of participants dropping out within the research context could improve long-term engagement in digital programs for the prevention of skin cancer.
This review has several strengths. Our study provided an initial analysis of the dropout from RCTs to prevent skin cancer through digital strategies. Our computed rates could help calculate sample sizes in future studies. We performed a sensitivity analysis that helped us detect outliers and confirm the absence of publication bias. Moreover, the subgroups and metaregression analyses allowed us to understand how loss of participants could be modified by different predictors.
The main limitation of our review is that potential literature from other databases with non-English records could have been missing. Furthermore, our outcomes may have been conditioned by the heterogeneity of the experimental interventions in the included studies. Some of the studies compared digital strategies with no intervention, so we cannot assert that dropouts from these groups could be related to external factors. Evidence from the subgroup meta-analysis sorted by an active comparator group should be interpreted with caution because of the low number of analyzed studies; further research is needed to obtain strong evidence. We were unable to propose tailored advice to improve retention for this kind of RCT owing to the sparse information on reasons for dropout provided by the authors.
This systematic review and meta-analysis calculated an overall pooled participant dropout rate of 9.5% (95% CI 5.0-17.5), which should be considered in the calculation of sample size in RCTs aimed at preventing skin cancer using digital health interventions. Although a slightly higher pooled dropout rate was recorded for digital strategies, the OR-based meta-analysis did not show significant differences against the comparator groups. Our meta-analyses of subgroups sorted by digital and comparator interventions did not present significant statistical differences. Age, sex, length of the intervention, and sample size did not modify the effect size, so they were not moderators of dropout. We highlight the need to follow the guidelines and standardize reporting of the number of and reasons for participants’ dropout because this will be the only effective way to design a successful plan to reduce the loss of participants in studies that analyze digital approaches to prevent skin cancer.
Supplemental material.
odds ratio
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
International Prospective Register of Systematic Reviews
The Cochrane Risk of Bias tool version 2
randomized clinical trial
None declared.