JMIR J Med Internet Res Journal of Medical Internet Research 1438-8871 JMIR Publications Toronto, Canada v24i9e39910 36083626 10.2196/39910 Original Paper Original Paper Enrollment and Retention of Participants in Remote Digital Health Studies: Scoping Review and Framework Proposal Leung Tiffany Morgan Dr. Susan Tizzani Michele Reuter Katja Daniore Paola MSc 1 2 https://orcid.org/0000-0003-3319-1125 Nittas Vasileios PhD 3 https://orcid.org/0000-0002-6685-8275 von Wyl Viktor Prof Dr 1
Institute for Implementation Science in Healthcare University of Zurich Universitatstrasse 84 Zurich, 8006 Switzerland 41 0446343762 viktor.vonwyl@uzh.ch
2 3 https://orcid.org/0000-0002-8754-9797 Institute for Implementation Science in Healthcare University of Zurich Zurich Switzerland Digital Society Initiative University of Zurich Zurich Switzerland Epidemiology, Biostatistics and Prevention Institute University of Zurich Zurich Switzerland Corresponding Author: Viktor von Wyl viktor.vonwyl@uzh.ch 9 2022 9 9 2022 24 9 e39910 27 5 2022 21 6 2022 12 7 2022 31 7 2022 ©Paola Daniore, Vasileios Nittas, Viktor von Wyl. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 09.09.2022. 2022

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

 Background

Digital technologies are increasingly used in health research to collect real-world data from wider populations. A new wave of digital health studies relies primarily on digital technologies to conduct research entirely remotely. Remote digital health studies hold promise to significant cost and time advantages over traditional, in-person studies. However, such studies have been reported to typically suffer from participant attrition, the sources for which are still largely understudied.

 Objective

To contribute to future remote digital health study planning, we present a conceptual framework and hypotheses for study enrollment and completion. The framework introduces 3 participation criteria that impact remote digital health study outcomes: (1) participant motivation profile and incentives or nudges, (2) participant task complexity, and (3) scientific requirements. The goal of this study is to inform the planning and implementation of remote digital health studies from a person-centered perspective.

 Methods

We conducted a scoping review to collect information on participation in remote digital health studies, focusing on methodological aspects that impact participant enrollment and retention. Comprehensive searches were conducted on the PubMed, CINAHL, and Web of Science databases, and additional sources were included in our study from citation searching. We included digital health studies that were fully conducted remotely, included information on at least one of the framework criteria during recruitment, onboarding or retention phases of the studies, and included study enrollment or completion outcomes. Qualitative analyses were performed to synthesize the findings from the included studies.

 Results

We report qualitative findings from 37 included studies that reveal high values of achieved median participant enrollment based on target sample size calculations, 128% (IQR 100%-234%), and median study completion, 48% (IQR 35%-76%). Increased median study completion is observed for studies that provided incentives or nudges to extrinsically motivated participants (62%, IQR 43%-78%). Reducing task complexity for participants in the absence of incentives or nudges did not improve median study enrollment (103%, IQR 102%-370%) or completion (43%, IQR 22%-60%) in observational studies, in comparison to interventional studies that provided more incentives or nudges (median study completion rate of 55%, IQR 38%-79%). Furthermore, there were inconsistencies in measures of completion across the assessed remote digital health studies, where only around half of the studies with completion measures (14/27, 52%) were based on participant retention throughout the study period.

 Conclusions

Few studies reported on participatory factors and study outcomes in a consistent manner, which may have limited the evidence base for our study. Our assessment may also have suffered from publication bias or unrepresentative study samples due to an observed preference for participants with digital literacy skills in digital health studies. Nevertheless, we find that future remote digital health study planning can benefit from targeting specific participant profiles, providing incentives and nudges, and reducing study complexity to improve study outcomes.

 remote digital health studies remote clinical trials remote cohorts digital epidemiology digital health health outcome conceptual framework user-centered design population-based digital health participant recruitment interventional study Introduction Background

The widespread availability of smartphones (estimated to be 3.6 billion users worldwide [1]) presents the opportunity to involve diverse population groups in health research. Mobile technologies, such as smartphones and wearables, have come to play a central role in health research, giving rise to digital health studies that are conducted partly or entirely remotely. Although there is no unified definition, we define remote digital health studies as longitudinal studies that use mobile technologies to conduct all key steps of a study completely online [2]. Remote digital health studies promise significant cost, time, and scalability advantages when compared with traditional studies, by allowing key steps of the study investigations to be conducted in real-time and without in-person presence [3,4]. Overcoming the barriers of time and physical presence, remote digital health studies allow for the long-term monitoring of larger populations and thus promise to advance health research and patient care delivery [5-7].

Despite these opportunities, recent studies report high participant attrition rates, likely partially attributable to the lack of in-person interactions between researchers and study participants. Other studies highlight the risk of recruitment bias, especially with younger, more affluent, and often healthier populations being overrepresented in studies with digital technologies [8-10]. These concerns point toward a possible imbalance between participants who typically join remote digital health studies and participants who are often underrepresented, but may benefit the most from remote digital health research and monitoring. This may be a result from a lack of understanding of the motivators, facilitators, and barriers that enable participation in remote digital health studies [11,12].

Trends of participant enrollment and retention have been widely investigated in traditional research settings [13-15] and with digital health studies following Eysenbach’s Law of Attrition [16]. However, there is a paucity of evidence that supports study planning in remote digital health research. Furthermore, most study planning recommendations for remote digital health studies are based on qualitative methods and focus on scientific, rather than participant-specific requirements [17,18]. This presents an unmet need for quantitative, evidence-based guidance that informs remote digital health study planning to enable high enrollment and retention through a person-centered lens.

 Aims

This review aims to explore participant enrollment and retention for remote digital health studies. We introduce a framework on the interplay between 3 criteria explored and validated in previous digital health studies: (1) participant motivation profile and incentives or nudges, (2) participant task complexity, and (3) scientific requirements. We propose hypotheses and explore them with a scoping review of remote digital health studies specifically, focusing on methodological aspects that affect participant enrollment and retention. The goal of this scoping review is to inform the planning and implementation of remote digital health studies from a person-centered perspective.

 Conceptual Framework and Hypotheses

We introduce a conceptual framework that encompasses the main factors that affect digital health study enrollment and retention from a person-centered lens, to guide the extraction of relevant information. The framework is based on the notion that enrollment and retention in remote digital health studies are influenced by 3 elements: (1) participant motivation profile and incentives or nudges, (2) participant task complexity, and (3) scientific requirements (Figure 1). Our conceptual framework and hypotheses were informed by Eysenbach [16], previous large-scale remote digital health studies [5,19], as well as by our personal experiences in planning and conducting 2 longitudinal remote digital health studies [20,21]. A more detailed description of the framework development process can be found in Multimedia Appendix 1 [2,5,14,16,19-28].

We define hypotheses to explore in this study. Specifically, we expect that incentives and nudges increase participant motivation to enroll in and complete a study (Hypothesis 1). On the contrary, we expect a decrease in enrollment and study completion with increased complexity of study tasks (Hypothesis 2a). Finally, we also expect that participants in interventional studies may be willing to endure higher task complexity than participants enrolled in observational studies (Hypothesis 2b).

Guiding framework for remote digital health studies.

 Conceptual Framework Definitions Motivation Profile of Participants and Offered Incentives

Participants’ motivation to enroll and complete remote digital health studies can be intrinsic or extrinsic. Intrinsic participation may be motivated by altruistic motives or by hopes for medical advances, especially among individuals with chronic diseases. Extrinsic participation can be motivated, among others, by investigators in the form of monetary incentives or clinical referrals [29,30]. The provision of incentives or nudges may help extrinsically motivated participants to enroll and participate in a study. Incentives are predominantly offered in the form of monetary compensation, while nudges mainly come in the form of reminders or personal contact [31-34]. Intrinsically motivated participants do not necessarily require incentives or nudges to enroll or participate in a study. It is to be noted, however, that intrinsic and extrinsic motivations are not necessarily mutually exclusive from one another and may coincide in a study. In this review, we conduct an exploratory assessment based on the aforesaid definitions. Different approaches to assessing participant motivations may exist.

 Complexity of Tasks Required From Participants

The frequency and complexity of tasks required from the participants, along with the expected duration of the study, can impact study completion. Tasks can be categorized as physical and mental tasks. Typical physical tasks required of participants include physical activity tasks, such as walking a certain number of steps every day, as well as other essential tasks to fulfill the study’s requirements, such as signing an informed consent form, filling out questionnaires at baseline or at follow-up, or logging of health-related outcomes [27,35,36]. Mental tasks, such as the measure of cognitive burden of a participant, are harder to quantify as they typically rely on patient-reported outcome measures [37,38]. In digital heath studies, task complexity is compounded by electronic measurements and device handling, which may require high digital literacy skills [39-41]. Task complexity can be reduced by accompanying measures, such as passive data collection, or technical support. Applied to remote digital health studies, high-complexity tasks come in the form of many required tasks over long study periods. Provision of assistance during participant onboarding may improve digital literacy skills of participants, while also instilling a sense of trust between the participant and the researcher through personal interactions [42,43].

 Scientific Requirements of the Study

Scientific requirements define the study design and expected target sample size. Therefore, scientific requirements set the goal of the study, while task complexity or incentives or nudges enable study goal achievement. Target sample sizes are generally estimated through statistical power analyses for enrollment goals [27,44]. Consideration of participant requirements may help increase the statistical power and reduce selection bias of the study. However, sample size calculation methods often may not anticipate participant losses to follow-up and failure to complete tasks [45]. Sensitivity analyses may be conducted after the study to assess the impact of deviations in participant enrollment or retention.

 Methods Study Outcome Definitions

In our study we refer to 3 phases of participants’ involvement in remote digital health studies: recruitment, onboarding, and retention (Tables 1 and 2). We summarize the outcomes of the 3 phases of participant involvement in remote digital health studies as study enrollment and completion. A detailed approach for defining each phase of the study and the outcomes can be found in Multimedia Appendix 1.

The 3 criteria in our proposed framework can affect all phases of remote digital health studies. In the next sections, we will explore our hypotheses by assessing the outcomes of interest for each framework criteria across all 3 phases of remote digital health studies.

Phases of digital health studies.

Phases Definition
Recruitment Fulfillment of requirements for study enrollment
Onboarding Provision of (technical) assistance to start study tasks
Retention Fulfillment of requirements for study completion

Outcomes of digital health studies.

Outcomes Measure
Study enrollment target (Achieved enrollment/target enrollment) × 100%
Study completion Percentage of enrolled participants who completed the study
Search Strategy and Study Selection

To explore our hypotheses, we conducted a scoping review according to the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-analyses Extension for Scoping Reviews) checklist [46] (Multimedia Appendix 2). Our search was performed on the PubMed, CINAHL, and Web of Science databases for primary research articles published between January 1, 2016, and June 31, 2021. We limited our search to this period based on the results of a preliminary search revealing a paucity of remote digital health studies published before 2016 [19,47]. We also assessed reviews and included relevant primary studies for a full-text review based on citation searches. The complete search strategy for each database can be found in Multimedia Appendix 3.

Our selection was guided by the criteria outlined in Textbox 1. Screening was conducted in 2 phases. Initially, we screened titles and abstracts and then the full texts. For both phases, the entire screening was conducted by one investigator (PD), while a second investigator (VN) performed checks on a randomly selected sample of studies in the title and abstract screening (80/662 articles, 12.1%) and in the full-text screening (50/150, 33.3%). Any disagreements were discussed and, if required, consensus was achieved through the third investigator (VvW). Agreement was 75/80 articles (94%) for title and abstract screening and 45/50 articles (90%) for full-text screening.

Literature inclusion criteria.

Studies that match the definition of a remote digital health study (ie, digital health study where all steps are conducted online and without in-person interactions between participants and study investigators).

Studies that mention their approach to recruit, onboard, or retain participants.

Studies that mention approaches to at least one of the proposed framework criteria.

Studies that provide evidence on study enrollment or study completion.

 Data Extraction and Synthesis

Data extraction was standardized yet developed iteratively. In cases where research articles referenced to the original protocol for the same study, the additional protocols were assessed to collect missing information of interest. The initial data extraction was based on standard study characteristics (eg, study design, participant characteristics) and guided by the conceptual framework (Figure 1). During the full-text screening, other criteria of interest (eg, measure of study completion) were identified as relevant and retrospectively included.

We conducted qualitative analyses to explore our hypotheses. Qualitative analyses are presented for the entire study sample, as well as for samples stratified based on the median study duration. Descriptions of qualitative data were provided to summarize key findings from the included studies within the structure of the conceptual framework. We also conducted an exploratory quantitative assessment of the framework criteria with the study enrollment and completion outcomes. The correlations between measures relevant to our study’s framework criteria and the study enrollment and completion outcomes retrieved from the included studies were assessed using Spearman rank correlation for continuous variables and the Kruskal-Wallis rank-sum test for categorical variables.

All screening and extraction procedures were completed in MS Excel (Microsoft, Inc.). All statistical analyses were completed in R, version 4.0.0 (R Foundation for Statistical Computing) using ggplot2, version 3.3.3, for plots. The threshold for statistical significance was set at P<.05 (2-tailed testing).

 Data Availability

The data from the papers that support the findings of this study are publicly available. All data used in this review can be found in Multimedia Appendices 4-6 [6,7,43,48-81].

 Results Data Search and Identification of Articles

Our database searches yielded 662 references (Figure 2). Twenty studies fulfilled all inclusion criteria and were finally included. An additional 38 studies were identified through citation searches in relevant systematic reviews, of which 17 met all inclusion criteria. Overall, 37 studies were included for assessment in this review.

Flowchart for study identification, screening, and inclusion.

 General Description of Included Studies

The study characteristics, study requirements, and framework criteria outcomes of the final study sample are presented in Multimedia Appendix 1 and summarized in Multimedia Appendices 4-6 [6,7,43,48-81], respectively. Of the 37 included studies, 19/37 (51%) were randomized control trials, 11/37 (30%) nonexperimental studies, and 7/37 (19%) quasi-experimental studies. All studies were longitudinal, 28/37 (76%) of which were interventional and 9/37 (24%) observational studies.

The most prevalent therapeutic areas in our study sample were psychiatry (11/37 studies, 30%), neurology (4/37 studies, 11%), and addiction (4/37 studies, 11%). Most studies (12/37, 32%) measured physical activity levels, 3/37 (8%) measured smoking cessation, 3/37 (8%) measured depression management, 2/37 (5%) measured stress management, 2/37 (5%) measured pain management, and the rest of the studies (15/37, 41%) measured other outcomes. Most studies relied on smartphones for the study procedure (23/37, 62%) and predominantly measured step counts (7/37, 19%). Most studies (34/37, 92%) collected patient-reported outcomes.

Most studies (25/37, 68%) targeted participants with an existing health condition, mainly enrolled females (median female enrollment of 77%, IQR 52%-88%) and participants with a median age of 39 years (IQR 35-47 years). A median of 43% (IQR 32-49%) of enrolled participants had a lower educational background, while information on enrolled participant income (27/37, 73%) and employment status (25/37, 68%) was largely missing from the studies.

 Description of Included Studies According to Framework

A description of the included studies based on our framework’s criteria is found in Table 3. The table summarizes the participants’ motivation profiles, the most prevalent incentives and nudges provided at each stage of the study (more information is available in Multimedia Appendix 6), the frequencies of the tasks for the studies and measures provided to reduce participant burden, the target sample size of the studies, and the study outcomes. Given the scarcity of mental task measures in the assessed studies, our review only included measures of physical tasks. The table is stratified by the median duration of the included studies (12 weeks, IQR 12-26 weeks).

A total of 27 studies reported on study completion (Multimedia Appendix 5); 14/27 (52%) studies assessed for completion based on participant retention throughout the study period, 10/27 (37%) studies assessed for completion based on researcher-defined metrics (eg, completion of 1 task within a 30-day period) and 3/27 (11%) studies assessed for completion based on the fulfillment of all study tasks.

In the next sections, study enrollment and completion are assessed based on our study’s hypotheses. An assessment of the task frequencies and study durations is made against measures provided to reduce participant burden, as well as incentives or nudges provided to participants. Further descriptions and correlations between the studies’ framework criteria and outcomes are reported in Multimedia Appendix 7.

Summary statistics of included studies based on framework criteria.

Profile Study durationa: ≤12 weeks (n=19) Study duration: >12 weeks (n=15) Study duration: unknown duration (n=3) Overall (n=37)
Motivation profile of participants and offered incentives or nudges
Motivation profiles, n (%)
Extrinsic 14 (74) 8 (53) 0 (0) 22 (59)
Intrinsic 5 (26) 7 (47) 3 (100) 15 (41)
Incentives/nudges: recruitmentb, n (%)
Monetary 4 (21) 3 (20) 2 (67) 9 (24)
Referral source 3 (16) 1 (7) 0 (0) 4 (11)
Vested interest 0 (0) 2 (13) 1 (33) 3 (8)
Incentives/nudges: onboarding, n (%)
Personal assistance 1 (5) 5 (33) 0 (0) 6 (16)
Peer support 0 (0) 1 (7) 0 (0) 1 (3)
Incentives/nudges: retention, n (%)
Monetary 10 (53) 6 (40) 1 (33) 15 (41)
Reminders 10 (53) 5 (33) 0 (0) 15 (41)
Personal contact 9 (47) 3 (20) 0 (0) 12 (32)
Incentives/nudges: minimum one phasec, n (%) 18 (95) 13 (87) 2 (67) 33 (89)
Complexity of tasks required from participants
Task complexity: study tasks
Monthly steps, median (IQR)d 16 (5-30) 30 (29-36) —e 28 (8-31)
Total steps, median (IQR)f 35 (12-93) 99 (14-180) 3.00 (3.0-3.0) 58 (10-120)
Task complexity: reduction of burden, n (%)
Passive monitoring 7 (37) 9 (60) 0 (0) 16 (43)
Short, repetitive daily tasks 6 (32) 8 (53) 0 (0) 14 (38)
At least one burden reductiong 9 (47) 12 (80) 0 (0) 21 (57)
Scientific requirements of the study
Target sample size, median (IQR) 72 (50-120) 313 (238-838) 473 (336-609) 200 (50-350)
Study outcomes
Enrolled participants, median (IQR) 281 (89-450) 560 (150-2800) 100 (55-200) 300 (89-950)
Retained participants, median (IQR) 110 (45-240) 800 (190-1700) — 180 (70-690)
Enrollment target (%), median (IQR) 150 (124-302) 101 (96-125) 82 (48-116) 128 (100-234)
Study completion (%), median (IQR) 48 (38-73) 55 (32-79) — 48 (35-76)

aThe duration of the study as defined in the study protocol.

bThe top 3 recruitment and retention incentives and nudges are reported; more information is available in Multimedia Appendix 6.

cIncentive or nudge provided in at least one of the study phases: recruitment, onboarding, or retention.

dThe number of physical tasks investigators required participants to do on a monthly basis throughout the study duration as defined in the study protocol.

eNot available.

fThe total number of physical tasks investigators required participants to do throughout the study duration as defined in the study protocol.

gBurden reduction for participants either through 1 of the 2 approaches, passive monitoring or short, repetitive daily tasks, provided in a study.

 Evidence for Hypotheses Hypothesis 1: Study Outcomes Based on Motivation Profiles and Incentives or Nudges

In this section we link evidence from the studies’ task complexity with the participant motivation profile and incentives or nudges criteria of our framework to assess study enrollment and completion outcomes. The 15 studies that enrolled intrinsically motivated participants reached a median enrollment target of 137% (IQR 98%-226%), and the 22 studies that enrolled extrinsically motivated participants reached a median enrollment target of 126% (IQR 102%-213%). The median completion rate of studies that focused on intrinsically motivated participants was 41% (IQR 20%-49%), whereas those that included extrinsically motivated participants had a median study completion of 62% (IQR 43%-78%; Figure 3).

Studies that enrolled intrinsically motivated participants had a median duration of 14 weeks (IQR 12-26) and had participants complete a median of 30 (IQR 12-36) tasks per month. Approximately half of these studies (7/15, 47%) offered passive monitoring of health data or had participants complete lower complexity, repetitive daily tasks (7/15, 47%).

Studies that enrolled extrinsically motivated participants had a median duration of 12 weeks (IQR 12-14) and had participants complete a median of 26 (IQR 7-30) tasks per month. Some of the studies (9/22, 41%) offered passive monitoring of health data and fewer offered participants lower complexity, repetitive daily tasks (7/22, 32%).

Incentives or nudges for recruitment of intrinsically motivated participants were offered only in 4/15 (27%) studies and incentives or nudges for retention were offered in 9/15 (60%) studies. For studies that targeted extrinsically motivated participants, incentives or nudges for recruitment were offered in 12/22 (55%) studies and incentives or nudges for retention were offered in all (22/22, 100%) studies.

Statistical significance testing did not reveal evidence for an effect of intrinsic or extrinsic study motivations or the provision of incentives and nudges on study enrollment or completion outcomes (Multimedia Appendix 7).

Study completion based on participant motivations and study requirements.

 Hypothesis 2: Study Outcomes Based on Task Complexity and Study Design

In this section we link evidence from the study task complexity with the scientific requirements criteria of our framework to assess study enrollment and completion outcomes.

Interventional studies achieved a median enrollment target of 128% (IQR 100%-150%) and observational studies a median enrollment target of 103% (IQR 102%-370%). Interventional studies achieved a median study completion of 55% (IQR 38%-79%) and observational studies achieved a median study completion of 43% (IQR 22%-60%; Figure 4).

A total of 28 interventional studies had a median duration of 12 weeks (IQR 12-14) and had participants complete a median of 16 (IQR 4-30) tasks per month. Few interventional studies (8/28, 29%) offered passive monitoring of health data and had participants complete lower complexity, repetitive daily tasks (8/28, 29%).

Nine observational studies had a median duration of 26 weeks (IQR 12-26 weeks) and required participants to complete a median of 30 (IQR 29-35) tasks per month. Most observational studies (7/9, 78%) offered passive monitoring of health data and had participants complete lower complexity, repetitive daily tasks (6/9, 67%).

Approximately half of the interventional studies (15/28, 54%) provided incentives or nudges for recruitment, and 26 interventional studies (26/28, 93%) provided incentives or nudges for retention. Only 1 observational study (1/9, 11%) provided an incentive for recruitment, while 5 (5/9, 56%) of the observational studies provided incentives or nudges for participant retention.

Statistical analyses did not reveal evidence for an effect of task complexity or study design on study enrollment or completion outcomes (Multimedia Appendix 7). The observed coefficients only suggest weak or no correlations, which were not statistically significant (Multimedia Appendix 7).

Study completion based on study design and requirements.

 Discussion Principal Findings

The studies from our sample were primarily interventional. Most studies targeted extrinsically motivated participants for shorter-length studies and provided incentives or nudges to recruit or retain participants. Around half of the studies provided measures to reduce participant burden through passive data collection or by requiring participants to complete frequent, shorter tasks. The study sample revealed high values of achieved target participant enrollment and retention. These findings suggest that the criteria defined in our framework may have an impact on the studies’ enrollment and completion outcomes.

Exploring our first hypothesis, high study enrollment is observed in studies that targeted intrinsically motivated participants. Study completion in these studies was lower than in those that targeted extrinsically motivated participants. This may be due to researchers requiring intrinsically motivated participants to complete more tasks per month in comparison to studies with extrinsically motivated participants or due the provision of additional incentives in studies with extrinsically motivated participants. Furthermore, the provision of lower complexity, daily repetitive tasks was similar between studies with intrinsically or extrinsically motivated participants. Our study’s preliminary qualitative analyses, although the results did not reach statistical significance, indicate that the provision of incentives or nudges as well as the reduction of required tasks from participants may contribute to higher study completion.

Exploring our second hypothesis, differences in study enrollment and completion outcomes are also observed between interventional and observational study designs. Here, higher enrollment is observed in interventional studies, despite higher efforts from most of the observational studies in our sample to achieve wider reach study recruitment and simplify enrollment procedures. Interventional studies revealed a higher study completion, although fewer measures to reduce participant burden were provided. We find that our framework provides preliminary relevant criteria and findings relevant to remote digital health study participation to guide researchers in study planning.

 Comparison With Prior Work Descriptive Assessment of Participation Strategies in Remote Digital Health Studies

Descriptive results from our study sample reveal a preference for investigators to provide incentives and nudges to retain rather than recruit participants. Researchers’ choice to focus on retaining participants could be in response to reported high attrition rates in digital health studies [15,83-85]. If participants are not retained, the quality of the evidence base of a study is decreased. From our study sample, the lower provision of incentives or nudges at recruitment could have likely been motivated by higher participant reach through open social networks, multimodal recruitment strategies, and user-friendly interfaces [48-64,80,81]. The replacement of in-person enrollment procedures with mobile-forward procedures that are easy to use, specifically, enables easier study enrollment, as mentioned in 8 of our studies [6,50,53,55-57,65,66]. However, it is important to consider that successful recruitment strategies do not necessarily translate into high participant retention. This could be due, in part, to unrepresentative samples that enroll in a study to collect financial incentives and then dropout [86,87]. Studies may be highly effective if they place equal importance on their recruitment and retention strategies, while applying sample validation approaches to ensure the representativeness of their study sample.

Studies of shorter durations reported lower study completion than those of longer durations. This is inconsistent with the observation that studies that ran for 12 weeks or less required less tasks to be completed by participants than in studies’ that ran for over 12 weeks. However, half of the longer duration studies reduced participant burden by asking them to complete repetitive, short-length tasks once a day [52,54,56,62,67-69,80]. For most of these studies, these tasks could be completed at any point in time during the day and the completion of all tasks was made optional. The majority of longer duration studies also collected data passively through smartphones or wearable sensors [7,48,51,54,60,61,67,70,71]. These findings are in line with recent literature on best practices to maximize participation in longitudinal digital health studies [88,89]. Specifically, evidence aligns with our observations that the integration of short, repetitive tasks that align with participants’ daily lives is more likely to achieve higher participation in comparison to longer, infrequent tasks with higher cognitive burden [49,72,90].

Few studies in our sample involved onboarding procedures for participants. This may be due to investigators’ possible preference for in-person interactions during study enrollment. Previous literature on this topic suggests that interacting with participants virtually, rather than in person, may contribute to higher study attrition [16,83,91]. However, a growing body of recent literature points to the potential of leveraging a combination of user-centered methodologies and easier user interfaces to personalize study designs and maintain high engagement through personal (remote) contact by allowing participants to build trust with researchers [92-95]. Furthermore, survey responses from 2 studies in our cohort that compared in-person and remote procedures reported high participant satisfaction with the remote interactions with investigators [43,51]. Nevertheless, a more thorough investigation on the role of in-person versus remote onboarding procedures is missing in the literature. This calls attention to a potentially essential step in remote digital health research that is understudied yet could significantly impact study participation.

 Exploration of Hypotheses From Relationships Between Framework Criteria and Outcomes

On average, the studies in our sample enrolled more participants than their target sample size calculations, with studies that ran for 12 weeks or less achieving higher enrollment targets than studies that ran for longer than 12 weeks. The median study completion rate was approximately 50%, which is relatively high when considering that retention rates as low as 10% are common in digital health research [7,28,96].

A description of our study sample provides indications of a possible higher study completion for studies that provide incentives or nudges. This was observed with interventional studies that, despite providing fewer lower complexity tasks from participants than the observational studies in our sample, managed to achieve higher study completion. The higher study completion could be due to interventional studies providing more incentives or nudges to participants than observational studies. The provision of incentives in studies is consistent with recent approaches that use willingness to accept estimations or incentive matching with local wages to enable higher study enrollment and completion [97-101]. Furthermore, the choice of offering nudges to study participants is supported by recent adaptations to the nudge theory, which claims that providing motivational elements in digital health research may affect decision making in study participants [102]. In recent studies, different forms of nudges, or motivational strategies, such as gamification, personalization of the digital solution, and peer support (eg, through citizen science methodologies [2]), have been reported as successful in maintaining high user retention [103-105]. The creation of online communities and support groups was also mentioned in 9 of our studies as a key contributor for participant retention [49,51,60,62,65,70,73,74,81].

Enrollment for interventional studies was higher than for observational studies. This was observed despite wider reach study recruitment and easier enrollment procedures efforts through digital recruitment channels and mobile technologies observed in most observational studies from our sample [49,54,61,75,76]. Observational studies also achieved lower completion outcomes despite providing tasks that reduce participant burden, which points to a possible lack of appropriate nudges to avoid participants dropping out after enrollment. It was also observed that studies focused on intrinsically motivated participants enrolled participants for longer studies and required them to complete more tasks, while providing less incentives or nudges than in studies focused on extrinsically motivated participants. This may have contributed to selection of specific participant profiles and higher loss to follow up, as evidenced by lower completion of studies with intrinsically motivated participants. Conversely, the studies with intrinsically motivated participants that achieved study completion values higher than the median of this group of studies predominantly focused on participants at risk or with chronic conditions in an observational study format [54,67,69,71,75]. This observation aligns with recent literature on the prominent role of disease status in enabling higher study retention in digital health studies through intrinsic motivations [19,106,107].

Our analyses yielded no statistical evidence to support our hypothesis of an interaction between factors that affect participation in remote digital health studies and the study outcomes. However, the lack of statistical support for the other criteria does not invalidate our hypotheses. First, only few remote digital health studies were identified that provided detailed information on approaches used to enhance study enrollment and retention in their methodologies. This warrants more systematic reporting of remote digital health study operations. Second, publication bias may have influenced our statistical calculations. For example, studies that failed to reach the target sample size may not have gotten published in peer-reviewed journals. This may have biased our correlations toward the null hypothesis. Third, there were observed preferences for the target participant profiles (eg, levels of digital literacy) and the study procedure that likely had an impact on the studies’ outcomes. To enable future assessments that evaluate whether specific participation factors, such as mental tasks that impact cognitive burden, have an impact on study outcomes, it is recommended to report these factors individually in remote digital health studies. The availability of these data could enable more comprehensive reviews that can thoroughly investigate these interactions through statistical analyses, as well as more in-depth explorations of participation enablers and inhibitors through digital survey studies.

 Inconsistencies of Study Completion Measures

Heterogeneity in researchers’ choices to measure study completion was observed in our sample. Although all the studies we assessed outlined the expected tasks for the study participants to complete in their methodologies (or in referenced larger studies or protocols [55,108-115]), only 3 measured study completion based on the achievement of all tasks defined in the study [64,77,81]. Interestingly, 9 studies measured study completion based on researcher-defined criteria for task completion (eg, completion of 1 task within a 30-day period) specific to their study design [49,53-55,60,61,73,75,80]. Given the novelty of digital health research, the choice of different study completion metrics by researchers is not surprising. However, this poses difficulties in allowing for direct comparisons between studies, as the former approach can be regarded as more conservative, while the latter more lenient. It is, therefore, important that study completion metrics are studied in more detail to determine whether a one-size-fits-all approach should be taken for all studies, or if bespoke approaches to measure completion are more suitable.

 Future Directions for Remote Digital Health Study Planning

Although our statistical analyses were inconclusive, we conducted an initial exploratory assessment of the study’s qualitative data. We adapted the conceptual framework based on best practices found from an assessment of our study sample’s descriptive results that can inform future planning of remote digital health studies:

Adapt incentives and nudges provided to participants based on their motivation profile: offer different incentives or nudges at each key step of the study procedure. Monetary incentives may contribute to higher study enrollment [48,53,55-57,70,78], after sample validation [86,87], whereas nudges in the form of assistance during onboarding [48,55] and the provision of reminders [50,53,55,60,64,73] or a participant community (eg, through citizen science [2]) could contribute to higher retention. As technology replaces in-person interactions, the procedures set in place should be user-friendly [50,53,55-57,65,66] and enable participants to build personal relationships, with either study participants or study investigators [116]. An adequate assessment of participant profiles and their motivations to join the study can help adapt the provision of incentives or nudges. It is recommended that study investigators do not underestimate the requirement for additional incentives or nudges with intrinsically motivated participants in their study planning.

Reduce and simplify the study’s tasks: reducing the number of tasks required from participants in combination with providing incentives may achieve higher enrollment and completion in studies with intrinsically motivated participants, especially if they align with participants’ daily lives [53-55,60,78,79]. The use of digital tools with simpler user interfaces and passive monitoring can also facilitate task completion by reducing participants’ cognitive burden [49,54,60,73,78,79]. The implementation of onboarding procedures may also increase participants’ trust, understanding of, and confidence in the study [43,51]. Based on our preliminary assessment of the upper quantile of our results, the required number of physical tasks in studies with intrinsically motivated participants should ideally not exceed 36 steps per month.

Broaden the scientific requirements: adjust the design of the investigation and target sample size by simplifying the study’s research question. Broader research questions that affect larger population groups may help with achieving a representative study sample, which can be generalized to wider populations [49,54,61,67,75]. Study designs that consider participant motivations, the provision of incentives or nudges, and task complexities can contribute to higher study enrollment and completion outcomes.

We also suggest more systematic reporting on these criteria for study enrollment and completion to facilitate further quantitative assessments and knowledge exchange. This is particularly relevant because, compared with traditional health studies, remote digital health studies bring upon additional, less well-studied challenges. To facilitate the development of remote digital health study methodologies, revised study planning approaches, as voiced previously by others [16], are necessary. This is also of relevance for hybrid or fully decentralized trials, where their reliance on remote procedures also affects participation [117,118]. There is also a need for systematic reporting of additional procedural aspects of study execution with remote digital health studies. Specifically, different reporting requirements are encouraged, such as adapted sample size calculations, approaches to remote recruitment, and onboarding of participants in the “Methods” sections of studies, as well as detailed participant descriptions and aligned reporting of study completion measures in the “Results” sections of studies. The formulation of reporting guidelines, similar to STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) [119], mERA (mobile health [mHealth] evidence reporting and assessment) guidelines [120], and CONSORT-EHEALTH standards [27], as well as further refining our conceptual framework with more evidence as it is made available, could be a first step in this direction.

 Strengths and Limitations

To our knowledge, our study is the first in the field that provides a comprehensive overview of the facilitators and barriers to participation in remote digital health studies. Our study is also the first to provide evidence-based guidelines to inform future remote digital health study planning. However, our study has limitations. First, there is no unified definition of a remote digital health study. As such, a broader definition could have yielded more studies of interest from our search strategy. Second, the analysis is reliant on inconsistent criteria for the outcomes of interest defined by the investigators of our study sample, such as approaches to calculate study completion. In this study, we grouped these criteria into 3 categories of outcome measures, however, a comparison of study outcomes with full accuracy was not possible. Third, the studies are classified based on criteria and assumptions defined by the investigators (eg, definition of task complexity), which we attempted to keep as broad as possible based on available literature in the field. Nevertheless, other variables as well as measures for the classifications and definitions could be possible. Fourth, the analyses were conducted based on a limited set of hypotheses defined in this study; more hypotheses could also be possible. Lastly, our study’s dual review approach, based on random screenings, instead of a complete dual screening, may have led to missing studies in our final study sample.

 Conclusion

In this study, we describe a conceptual framework to introduce criteria that affect remote digital health study participation from a person-centered lens. We apply this framework to remote digital health studies to explore hypotheses on the factors that affect participation outcomes. The compiled data from our scoping review reveal that targeting specific participant profiles, the provision of incentives and nudges, or the reduction of study complexity at any stage of the study may improve study outcomes. Future remote digital health study planning requires a focus on participant requirements, as well as broadening scientific requirements to increase participation in studies. Our proposed framework provides an initial structure to facilitate remote digital health study planning, but we highlight the need for systematic reporting guidelines to enable further assessments and knowledge exchange.

Methods and additional descriptive results.

PRISMA-ScR checklist. PRISMA-ScR: Preferred Reporting Items for Systematic reviews and Meta-analyses Extension for Scoping Reviews.

Search strategy. Also see Multimedia Appendix 2.

General characteristics of included studies.

Requirements and outcomes of included studies.

Incentives or nudges and task complexity of included studies.

Associations between framework criteria and outcomes.

 Abbreviations CONSORT-EHEALTH

Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth

 mERA

mobile health (mHealth) evidence reporting and assessment

 PRISMA-ScR

Preferred Reporting Items for Systematic reviews and Meta-analyses Extension for Scoping Reviews

 STROBE

STrengthening the Reporting of OBservational studies in Epidemiology

PD wrote the initial plan for the review, conducted the literature search, carried out the analysis, wrote the first draft of the paper, and wrote and revised the final manuscript. VN assisted with the initial plan for the review, conducted the literature search, helped interpret the findings, and revised and approved the final manuscript. VvW conceptualized the study, helped interpret the findings, and revised and approved the final manuscript.

None declared.

 Global Smartphones Sales May Have Peaked Internet International Monetary Fund 2021-05-04 https://www.imf.org/en/Publications/WP/Issues/2020/05/29/Global-Smartphones-Sales-May-Have-Peaked-49361 Puhan MA Steinemann N Kamm CP Müller S Kuhle J Kurmann R Calabrese P Kesselring J von Wyl V Swiss Multiple Sclerosis Registry Smsr A digitally facilitated citizen-science driven approach accelerates participant recruitment and increases study population diversity Swiss Med Wkly 2018 05 16 148 w14623 10.4414/smw.2018.14623 29767828 smw-14623 Anguera JA Jordan JT Castaneda D Gazzaley A Areán PA Conducting a fully mobile and randomised clinical trial for depression: access, engagement and expense BMJ Innov 2016 01 2 1 14 21 10.1136/bmjinnov-2015-000098 27019745 bmjinnov-2015-000098 PMC4789688 Inan OT Tenaerts P Prindiville SA Reynolds HR Dizon DS Cooper-Arnold K Turakhia M Pletcher MJ Preston KL Krumholz HM Marlin BM Mandl KD Klasnja P Spring B Iturriaga E Campo R Desvigne-Nickens P Rosenberg Y Steinhubl SR Califf RM Digitizing clinical trials NPJ Digit Med 2020 3 101 10.1038/s41746-020-0302-y 32821856 302 PMC7395804 Perez MV Mahaffey KW Hedlin H Rumsfeld JS Garcia A Ferris T Balasubramanian V Russo AM Rajmane A Cheung L Hung G Lee J Kowey P Talati N Nag D Gummidipundi SE Beatty A Hills MT Desai S Granger CB Desai M Turakhia MP Apple Heart Study Investigators Large-Scale Assessment of a Smartwatch to Identify Atrial Fibrillation N Engl J Med 2019 11 14 381 20 1909 1917 10.1056/NEJMoa1901183 31722151 PMC8112605 Bot BM Suver C Neto EC Kellen M Klein A Bare C Doerr M Pratap A Wilbanks J Dorsey ER Friend SH Trister AD The mPower study, Parkinson disease mobile data collected using ResearchKit Sci Data 2016 03 03 3 160011 10.1038/sdata.2016.11 26938265 sdata201611 PMC4776701 Crouthamel M Quattrocchi E Watts S Wang S Berry P Garcia-Gancedo L Hamy V Williams RE Using a ResearchKit Smartphone App to Collect Rheumatoid Arthritis Symptoms From Real-World Participants: Feasibility Study JMIR Mhealth Uhealth 2018 09 13 6 9 e177 10.2196/mhealth.9656 30213779 v6i9e177 PMC6231853 Nouri SS Adler-Milstein J Thao C Acharya P Barr-Walker J Sarkar U Lyles C Patient characteristics associated with objective measures of digital health tool use in the United States: A literature review J Am Med Inform Assoc 2020 05 01 27 5 834 841 10.1093/jamia/ocaa024 32364238 5828196 PMC7309253 Langford A Orellana K Kalinowski J Aird C Buderer N Use of Tablets and Smartphones to Support Medical Decision Making in US Adults: Cross-Sectional Study JMIR Mhealth Uhealth 2020 08 12 8 8 e19531 10.2196/19531 32784181 v8i8e19531 PMC7450375 Langford AT Solid CA Scott E Lad M Maayan E Williams SK Seixas AA Mobile Phone Ownership, Health Apps, and Tablet Use in US Adults With a Self-Reported History of Hypertension: Cross-Sectional Study JMIR Mhealth Uhealth 2019 01 14 7 1 e12228 10.2196/12228 31344667 v7i1e12228 PMC6682274 Sezgin E Özkan-Yildirim S Yildirim S Investigation of physicians' awareness and use of mHealth apps: A mixed method study Health Policy and Technology 2017 09 6 3 251 267 10.1016/j.hlpt.2017.07.007 Lennon MR Bouamrane M Devlin AM O'Connor S O'Donnell C Chetty U Agbakoba R Bikker A Grieve E Finch T Watson N Wyke S Mair FS Readiness for Delivering Digital Health at Scale: Lessons From a Longitudinal Qualitative Evaluation of a National Digital Health Innovation Program in the United Kingdom J Med Internet Res 2017 02 16 19 2 e42 10.2196/jmir.6900 28209558 v19i2e42 PMC5334516 Neve MJ Collins CE Morgan PJ Dropout, nonusage attrition, and pretreatment predictors of nonusage attrition in a commercial Web-based weight loss program J Med Internet Res 2010 12 14 12 4 e69 10.2196/jmir.1640 21156470 v12i4e69 PMC3056525 Pedersen DH Mansourvar M Sortsø C Schmidt T Predicting Dropouts From an Electronic Health Platform for Lifestyle Interventions: Analysis of Methods and Predictors J Med Internet Res 2019 09 04 21 9 e13617 10.2196/13617 31486409 v21i9e13617 PMC6753691 Van der Mispel C Poppe L Crombez G Verloigne M De Bourdeaudhuij I A Self-Regulation-Based eHealth Intervention to Promote a Healthy Lifestyle: Investigating User and Website Characteristics Related to Attrition J Med Internet Res 2017 07 11 19 7 e241 10.2196/jmir.7277 28698168 v19i7e241 PMC5527252 Eysenbach G The law of attrition J Med Internet Res 2005 03 31 7 1 e11 10.2196/jmir.7.1.e11 15829473 v7e11 PMC1550631 Global strategy on digital health 2020-2025 World Health Organization 2021 Geneva World Health Organization Chu L Shah A Rouholiman D Riggare S Gamble J Patient-Centric Strategies in Digital Health Digital Health. Health Informatics 2018 Cham Springer International Publishing Pratap A Neto EC Snyder P Stepnowsky C Elhadad N Grant D Mohebbi MH Mooney S Suver C Wilbanks J Mangravite L Heagerty PJ Areán P Omberg L Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants NPJ Digit Med 2020 2 17 3 1 21 10.1038/s41746-020-0224-8 32128451 224 PMC7026051 Corona Immunitas 2021-06-24 https://www.corona-immunitas.ch/ Steinemann N Kuhle J Calabrese P Kesselring J Disanto G Merkler D Pot C Ajdacic-Gross V Rodgers S Puhan MA von Wyl V Swiss Multiple Sclerosis Registry The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research BMC Neurol 2018 08 13 18 1 111 10.1186/s12883-018-1118-0 30103695 10.1186/s12883-018-1118-0 PMC6088400 Berwick Donald M Disseminating innovations in health care JAMA 2003 04 16 289 15 1969 75 10.1001/jama.289.15.1969 12697800 289/15/1969 Robinson Ja Kocman D Speyer O Gerasopoulos E Meeting volunteer expectations — a review of volunteer motivations in citizen science and best practices for their retention through implementation of functional features in CS tools Journal of Environmental Planning and Management 2021 02 02 64 12 2089 2113 10.1080/09640568.2020.1853507 Sieverink Floor Kelders Saskia M van Gemert-Pijnen Julia Ewc Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence J Med Internet Res 2017 12 06 19 12 e402 10.2196/jmir.8578 29212630 v19i12e402 PMC5738543 Christensen Helen Griffiths Kathleen M Farrer Louise Adherence in internet interventions for anxiety and depression J Med Internet Res 2009 04 24 11 2 e13 10.2196/jmir.1194 19403466 v11i2e13 PMC2762797 Heinsch Milena Wyllie Jessica Carlson Jamie Wells Hannah Tickner Campbell Kay-Lambkin Frances Theories Informing eHealth Implementation: Systematic Review and Typology Classification J Med Internet Res 2021 05 31 23 5 e18500 10.2196/18500 34057427 v23i5e18500 PMC8204232 Eysenbach G CONSORT-EHEALTH Group CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions J Med Internet Res 2011 12 31 13 4 e126 10.2196/jmir.1923 22209829 v13i4e126 PMC3278112 van Mierlo Trevor The 1% rule in four digital health social networks: an observational study J Med Internet Res 2014 02 04 16 2 e33 10.2196/jmir.2966 24496109 v16i2e33 PMC3939180 Milward J Drummond C Fincham-Campbell S Deluca P What makes online substance-use interventions engaging? A systematic review and narrative synthesis Digit Health 2018 02 4 2055207617743354 10.1177/2055207617743354 29942622 10.1177_2055207617743354 PMC6001270 de Ridder M Kim J Jing Y Khadra M Nanan R A systematic review on incentive-driven mobile health technology: As used in diabetes management J Telemed Telecare 2017 01 23 1 26 35 10.1177/1357633X15625539 26888421 1357633X15625539 Thaler R.H. Sunstein C.R. Nudge: Improving decisions about health, wealth, and happiness Const Polit Econ 2008 8 22 New Haven Yale University Press 356 360 Dai H Saccardo S Han MA Roh L Raja N Vangala S Modi H Pandya S Sloyan M Croymans DM Behavioural nudges increase COVID-19 vaccinations Nature 2021 09 02 597 7876 404 409 10.1038/s41586-021-03843-2 34340242 10.1038/s41586-021-03843-2 PMC8443442 Horne BD Muhlestein JB Lappé DL May HT Le VT Bair TL Babcock D Bride D Knowlton KU Anderson JL Behavioral Nudges as Patient Decision Support for Medication Adherence: The ENCOURAGE Randomized Controlled Trial Am Heart J 2022 02 244 125 134 10.1016/j.ahj.2021.11.001 34798073 S0002-8703(21)00444-0 Bauer B Tucker R Capron D A Nudge in a New Direction: Integrating Behavioral Economic Strategies Into Suicide Prevention Work Clinical Psychological Science 2019 05 01 7 3 612 620 10.1177/2167702618809367 Nittas V Puhan MA von Wyl V Toward a Working Definition of eCohort Studies in Health Research: Narrative Literature Review JMIR Public Health Surveill 2021 01 21 7 1 e24588 10.2196/24588 33475521 v7i1e24588 PMC7861999 Nebeker C Gholami M Kareem D Kim E Applying a Digital Health Checklist and Readability Tools to Improve Informed Consent for Digital Health Research Front Digit Health 2021 07 15 3 690901 10.3389/fdgth.2021.690901 34713167 PMC8521887 Torous J Michalak EE O'Brien HL Digital Health and Engagement-Looking Behind the Measures and Methods JAMA Netw Open 2020 07 01 3 7 e2010918 10.1001/jamanetworkopen.2020.10918 32678446 2768343 Taylor KI Staunton H Lipsmeier F Nobbs D Lindemann M Outcome measures based on digital health technology sensor data: data- and patient-centric approaches NPJ Digit Med 2020 7 23 3 1 97 10.1038/s41746-020-0305-8 32715091 305 PMC7378210 Mackert M Mabry-Flynn A Champlin S Donovan EE Pounders K Health Literacy and Health Information Technology Adoption: The Potential for a New Digital Divide J Med Internet Res 2016 10 04 18 10 e264 10.2196/jmir.6349 27702738 v18i10e264 PMC5069402 El Benny M Kabakian-Khasholian T El-Jardali F Bardus M Application of the eHealth Literacy Model in Digital Health Interventions: Scoping Review J Med Internet Res 2021 06 03 23 6 e23473 10.2196/23473 34081023 v23i6e23473 PMC8212628 Rowsell A Muller I Murray E Little P Byrne CD Ganahl K Müller G Gibney S Lyles CR Lucas A Nutbeam D Yardley L Views of People With High and Low Levels of Health Literacy About a Digital Intervention to Promote Physical Activity for Diabetes: A Qualitative Study in Five Countries J Med Internet Res 2015 10 12 17 10 e230 10.2196/jmir.4999 26459743 v17i10e230 PMC4642371 Figueroa CA Murayama H Amorim PC White A Quiterio A Luo T Aguilera A Smith ADR Lyles CR Robinson V von Vacano C Applying the Digital Health Social Justice Guide Front Digit Health 2022 4 807886 10.3389/fdgth.2022.807886 35295620 PMC8918521 Hernandez-Ramos R Aguilera A Garcia F Miramontes-Gomez J Pathak LE Figueroa CA Lyles CR Conducting Internet-Based Visits for Onboarding Populations With Limited Digital Literacy to an mHealth Intervention: Development of a Patient-Centered Approach JMIR Form Res 2021 04 29 5 4 e25299 10.2196/25299 33872184 v5i4e25299 PMC8086779 Barnett I Torous J Reeder HT Baker J Onnela J Determining sample size and length of follow-up for smartphone-based digital phenotyping studies J Am Med Inform Assoc 2020 12 09 27 12 1844 1849 10.1093/jamia/ocaa201 33043370 5920884 PMC7727382 Whitley E Ball J Statistics review 4: sample size calculations Crit Care 2002 08 6 4 335 341 10.1186/cc1521 12225610 PMC137461 Tricco AC Lillie E Zarin W O'Brien KK Colquhoun H Levac D Moher D Peters MD Horsley T Weeks L Hempel S Akl EA Chang C McGowan J Stewart L Hartling L Aldcroft A Wilson MG Garritty C Lewin S Godfrey CM Macdonald MT Langlois EV Soares-Weiser K Moriarty J Clifford T Tunçalp Ö Straus SE PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation Ann Intern Med 2018 10 02 169 7 467 473 10.7326/M18-0850 30178033 2700389 Loncar-Turukalo T Zdravevski E Machado da Silva J Chouvarda I Trajkovik V Literature on Wearable Technology for Connected Health: Scoping Review of Research Trends, Advances, and Barriers J Med Internet Res 2019 09 05 21 9 e14017 10.2196/14017 31489843 v21i9e14017 PMC6818529 Damschroder LJ Buis LR McCant FA Kim HM Evans R Oddone EZ Bastian LA Hooks G Kadri R White-Clark C Richardson CR Gierisch JM Effect of Adding Telephone-Based Brief Coaching to an mHealth App (Stay Strong) for Promoting Physical Activity Among Veterans: Randomized Controlled Trial J Med Internet Res 2020 08 04 22 8 e19216 10.2196/19216 32687474 v22i8e19216 PMC7435619 Bailey JF Agarwal V Zheng P Smuck M Fredericson M Kennedy DJ Krauss J Digital Care for Chronic Musculoskeletal Pain: 10,000 Participant Longitudinal Cohort Study J Med Internet Res 2020 05 11 22 5 e18250 10.2196/18250 32208358 v22i5e18250 Watson NL Mull KE Heffner JL McClure JB Bricker JB Participant Recruitment and Retention in Remote eHealth Intervention Trials: Methods and Lessons Learned From a Large Randomized Controlled Trial of Two Web-Based Smoking Interventions J Med Internet Res 2018 08 24 20 8 e10351 10.2196/10351 30143479 v20i8e10351 PMC6128955 Bott N Kumar S Krebs C Glenn JM Madero EN Juusola JL A Remote Intervention to Prevent or Delay Cognitive Impairment in Older Adults: Design, Recruitment, and Baseline Characteristics of the Virtual Cognitive Health (VC Health) Study JMIR Res Protoc 2018 08 13 7 8 e11368 10.2196/11368 30104186 v7i8e11368 Hamilton FL Hornby J Sheringham J Linke S Ashton C Moore K Stevenson F Murray E DIAMOND (DIgital Alcohol Management ON Demand): a feasibility RCT and embedded process evaluation of a digital health intervention to reduce hazardous and harmful alcohol use recruiting in hospital emergency departments and online Pilot Feasibility Stud 2018 4 114 10.1186/s40814-018-0303-7 29946479 303 PMC6003139 Abbate KJ Hingle MD Armin J Giacobbi P Gordon JS Recruiting Women to a Mobile Health Smoking Cessation Trial: Low- and No-Cost Strategies JMIR Res Protoc 2017 11 03 6 11 e219 10.2196/resprot.7356 29101091 v6i11e219 PMC5695655 Druce KL McBeth J van der Veer SN Selby DA Vidgen B Georgatzis K Hellman B Lakshminarayana R Chowdhury A Schultz DM Sanders C Sergeant JC Dixon WG Recruitment and Ongoing Engagement in a UK Smartphone Study Examining the Association Between Weather and Pain: Cohort Study JMIR Mhealth Uhealth 2017 11 01 5 11 e168 10.2196/mhealth.8162 29092810 v5i11e168 PMC5688244 Gordon JS Armin J Giacobbi P Cunningham JK Johnson T Abbate K Howe CL Roe DJ Development and evaluation of the See Me Smoke-Free multi-behavioral mHealth app for women smokers Transl Behav Med 2017 02 02 7 2 172 184 10.1007/s13142-017-0463-7 28155107 10.1007/s13142-017-0463-7 PMC5526811 Laws RA Litterbach EV Denney-Wilson EA Russell CG Taki S Ong K Elliott RM Lymer SJ Campbell KJ A Comparison of Recruitment Methods for an mHealth Intervention Targeting Mothers: Lessons from the Growing Healthy Program J Med Internet Res 2016 09 15 18 9 e248 10.2196/jmir.5691 27634633 PMC5043120 Ashford MT Olander EK Rowe H Fisher JR Ayers S Feasibility and Acceptability of a Web-Based Treatment with Telephone Support for Postpartum Women With Anxiety: Randomized Controlled Trial JMIR Ment Health 2018 04 20 5 2 e19 10.2196/mental.9106 29678804 v5i2e19 PMC5938691 Fleischmann RJ Harrer M Zarski A Baumeister H Lehr D Ebert DD Patients' experiences in a guided Internet- and App-based stress intervention for college students: A qualitative study Internet Interv 2018 06 12 130 140 10.1016/j.invent.2017.12.001 30135777 S2214-7829(17)30086-6 PMC6096317 Bidargaddi N Musiat P Winsall M Vogl G Blake V Quinn S Orlowski S Antezana G Schrader G Efficacy of a Web-Based Guided Recommendation Service for a Curated List of Readily Available Mental Health and Well-Being Mobile Apps for Young People: Randomized Controlled Trial J Med Internet Res 2017 05 12 19 5 e141 10.2196/jmir.6775 28500020 v19i5e141 PMC5446666 Edney S Ryan JC Olds T Monroe C Fraysse F Vandelanotte C Plotnikoff R Curtis R Maher C User Engagement and Attrition in an App-Based Physical Activity Intervention: Secondary Analysis of a Randomized Controlled Trial J Med Internet Res 2019 11 27 21 11 e14645 10.2196/14645 31774402 v21i11e14645 PMC6906621 Chan YY Wang P Rogers L Tignor N Zweig M Hershman SG Genes N Scott ER Krock E Badgeley M Edgar R Violante S Wright R Powell CA Dudley JT Schadt EE The Asthma Mobile Health Study, a large-scale clinical observational study using ResearchKit Nat Biotechnol 2017 04 35 4 354 362 10.1038/nbt.3826 28288104 nbt.3826 PMC5559298 Chernick L Improving Adolescent Sexual and Reproductive Health: Can Mobile Health Interventions Affect Behavior? Pediatrics 2021 03 147 3 e2020029801 10.1542/peds.2020-029801 33568492 peds.2020-029801 PMC7919106 Short CE Rebar A James EL Duncan MJ Courneya KS Plotnikoff RC Crutzen R Vandelanotte C How do different delivery schedules of tailored web-based physical activity advice for breast cancer survivors influence intervention use and efficacy? J Cancer Surviv 2017 02 11 1 80 91 10.1007/s11764-016-0565-0 27498099 10.1007/s11764-016-0565-0 Zarski A Lehr D Berking M Riper H Cuijpers P Ebert DD Adherence to Internet-Based Mobile-Supported Stress Management: A Pooled Analysis of Individual Participant Data From Three Randomized Controlled Trials J Med Internet Res 2016 06 29 18 6 e146 10.2196/jmir.4493 27357528 v18i6e146 PMC4945816 Poppe L Van der Mispel C Crombez G De Bourdeaudhuij I Schroé H Verloigne M How Users Experience and Use an eHealth Intervention Based on Self-Regulation: Mixed-Methods Study J Med Internet Res 2018 10 01 20 10 e10412 10.2196/10412 30274961 v20i10e10412 PMC6231831 Zlotorzynska M Bauermeister JA Golinkoff JM Lin W Sanchez TH Hightow-Weidman L Online recruitment of youth for mHealth studies Mhealth 2021 7 27 10.21037/mhealth-20-64 33898596 mh-07-20-64 PMC8063013 Garabedian LF Ross-Degnan D LeCates RF Wharam JF Uptake and use of a diabetes management program with a mobile glucometer Prim Care Diabetes 2019 12 13 6 549 555 10.1016/j.pcd.2019.03.010 31097343 S1751-9918(18)30393-0 Korinek EV Phatak SS Martin CA Freigoun MT Rivera DE Adams MA Klasnja P Buman MP Hekler EB Adaptive step goals and rewards: a longitudinal growth model of daily steps for a smartphone-based walking intervention J Behav Med 2018 02 41 1 74 86 10.1007/s10865-017-9878-3 28918547 10.1007/s10865-017-9878-3 Schneider RB Omberg L Macklin EA Daeschler M Bataille L Anthwal S Myers TL Baloga E Duquette S Snyder P Amodeo K Tarolli CG Adams JL Callahan KF Gottesman J Kopil CM Lungu C Ascherio A Beck JC Biglan K Espay AJ Tanner C Oakes D Shoulson I Novak D Kayson E Ray Dorsey E Mangravite L Schwarzschild MA Simuni T Parkinson Study Group AT-HOME PD Investigators Design of a virtual longitudinal observational study in Parkinson's disease (AT-HOME PD) Ann Clin Transl Neurol 2021 02 8 2 308 320 10.1002/acn3.51236 33350601 PMC7886038 Kim H Ray CD Veluscek AM Complementary Support from Facilitators and Peers for Promoting mHealth Engagement and Weight Loss J Health Commun 2017 11 22 11 905 912 10.1080/10810730.2017.1373876 29125393 Baca-Motes K Edwards AM Waalen J Edmonds S Mehta RR Ariniello L Ebner GS Talantov D Fastenau JM Carter CT Sarich TC Felicione E Topol EJ Steinhubl SR Digital recruitment and enrollment in a remote nationwide trial of screening for undiagnosed atrial fibrillation: Lessons from the randomized, controlled mSToPS trial Contemp Clin Trials Commun 2019 06 14 100318 10.1016/j.conctc.2019.100318 30656241 S2451-8654(18)30168-6 PMC6329362 Richards D Murphy T Viganó N Timulak L Doherty G Sharry J Hayes C Acceptability, satisfaction and perceived efficacy of "Space from Depression": an internet-delivered treatment for depression Internet Interv 2016 09 5 12 22 10.1016/j.invent.2016.06.007 30135802 S2214-7829(16)30034-3 PMC6096253 Schoenfelder E Moreno M Wilner M Whitlock KB Mendoza JA Piloting a mobile health intervention to increase physical activity for adolescents with ADHD Prev Med Rep 2017 06 6 210 213 10.1016/j.pmedr.2017.03.003 28373931 S2211-3355(17)30045-1 PMC5374871 Schlosser DA Campellone TR Truong B Anguera JA Vergani S Vinogradov S Arean P The feasibility, acceptability, and outcomes of PRIME-D: A novel mobile intervention treatment for depression Depress Anxiety 2017 06 34 6 546 554 10.1002/da.22624 28419621 PMC5634707 Pratap A Grant D Vegesna A Tummalacherla M Cohan S Deshpande C Mangravite L Omberg L Evaluating the Utility of Smartphone-Based Sensor Assessments in Persons With Multiple Sclerosis in the Real-World Using an App (elevateMS): Observational, Prospective Pilot Digital Health Study JMIR Mhealth Uhealth 2020 10 27 8 10 e22108 10.2196/22108 33107827 v8i10e22108 PMC7655470 Williamson GR O'Connor A Chamberlain C Halpin D mHealth resources for asthma and pregnancy care: Methodological issues and social media recruitment. A discussion paper J Adv Nurs 2018 10 74 10 2442 2449 10.1111/jan.13773 29943472 Blake H Suggs LS Coman E Aguirre L Batt ME Active8! Technology-Based Intervention to Promote Physical Activity in Hospital Employees Am J Health Promot 2017 03 31 2 109 118 10.4278/ajhp.140415-QUAN-143 26559712 Keadle SK Meuter L Phelan S Phillips SM Charity-based incentives motivate young adult cancer survivors to increase physical activity: a pilot randomized clinical trial J Behav Med 2021 10 44 5 682 693 10.1007/s10865-021-00218-w 33825070 10.1007/s10865-021-00218-w Mitchell M White L Lau E Leahey T Adams MA Faulkner G Evaluating the Carrot Rewards App, a Population-Level Incentive-Based Intervention Promoting Step Counts Across Two Canadian Provinces: Quasi-Experimental Study JMIR Mhealth Uhealth 2018 09 20 6 9 e178 10.2196/mhealth.9912 30148712 v6i9e178 PMC6231836 Pratap A Renn BN Volponi J Mooney SD Gazzaley A Arean PA Anguera JA Using Mobile Apps to Assess and Treat Depression in Hispanic and Latino Populations: Fully Remote Randomized Clinical Trial J Med Internet Res 2018 08 09 20 8 e10130 10.2196/10130 30093372 v20i8e10130 PMC6107735 Edney S Looyestyn J Ryan J Kernot J Maher C Posts, pics, or polls? Which post type generates the greatest engagement in a Facebook physical activity intervention? Transl Behav Med 2018 11 21 8 6 953 957 10.1093/tbm/iby006 29660093 4961537 Food and Drug Administration (FDA) Spectrum of Diseases/Conditions FDA 2000 1 28 2022-03-16 https://www.fda.gov/drugs/development-resources/spectrum-diseasesconditions Meyerowitz-Katz G Ravi S Arnolda L Feng X Maberly G Astell-Burt T Rates of Attrition and Dropout in App-Based Interventions for Chronic Disease: Systematic Review and Meta-Analysis J Med Internet Res 2020 09 29 22 9 e20283 10.2196/20283 32990635 v22i9e20283 PMC7556375 Lie SS Karlsen B Oord ER Graue M Oftedal B Dropout From an eHealth Intervention for Adults With Type 2 Diabetes: A Qualitative Study J Med Internet Res 2017 05 30 19 5 e187 10.2196/jmir.7479 28559223 v19i5e187 PMC5470008 Becker S Miron-Shatz T Schumacher N Krocza J Diamantidis C Albrecht U mHealth 2.0: Experiences, Possibilities, and Perspectives JMIR Mhealth Uhealth 2014 2 2 e24 10.2196/mhealth.3328 25099752 v2i2e24 PMC4114478 Pratt-Chapman M Moses J Arem H Strategies for the Identification and Prevention of Survey Fraud: Data Analysis of a Web-Based Survey JMIR Cancer 2021 07 16 7 3 e30730 10.2196/30730 34269685 v7i3e30730 PMC8325077 Glazer JV MacDonnell K Frederick C Ingersoll K Ritterband LM Liar! Liar! Identifying eligibility fraud by applicants in digital health research Internet Interv 2021 09 25 100401 10.1016/j.invent.2021.100401 34094883 S2214-7829(21)00041-5 PMC8164029 Kolovson S Pratap A Duffy J Allred R Munson SA Areán PA Understanding Participant Needs for Engagement and Attitudes towards Passive Sensing in Remote Digital Health Studies Int Conf Pervasive Comput Technol Healthc 2020 05 2020 347 362 10.1145/3421937.3422025 33717638 PMC7955667 Baumel A Muench FJ Effort-Optimized Intervention Model: Framework for Building and Analyzing Digital Interventions That Require Minimal Effort for Health-Related Gains J Med Internet Res 2021 03 12 23 3 e24905 10.2196/24905 33709943 v23i3e24905 PMC7998331 Druce KL Dixon WG McBeth J Maximizing Engagement in Mobile Health Studies: Lessons Learned and Future Directions Rheum Dis Clin North Am 2019 05 45 2 159 172 10.1016/j.rdc.2019.01.004 30952390 S0889-857X(19)30004-3 PMC6483978 Rawstorn JC Gant N Rolleston A Whittaker R Stewart R Benatar J Warren I Meads A Jiang Y Maddison R End Users Want Alternative Intervention Delivery Models: Usability and Acceptability of the REMOTE-CR Exercise-Based Cardiac Telerehabilitation Program Arch Phys Med Rehabil 2018 11 99 11 2373 2377 10.1016/j.apmr.2018.06.027 30076800 S0003-9993(18)30919-5 Partridge S Redfern J Strategies to Engage Adolescents in Digital Health Interventions for Obesity Prevention and Management Healthcare 2018 06 21 6 3 70 10.3390/healthcare6030070 29933550 PMC6163226 Tebb KP Leng Trieu S Rico R Renteria R Rodriguez F Puffer M A Mobile Health Contraception Decision Support Intervention for Latina Adolescents: Implementation Evaluation for Use in School-Based Health Centers JMIR Mhealth Uhealth 2019 03 14 7 3 e11163 10.2196/11163 30869649 v7i3e11163 PMC6437609 Dulli L Ridgeway K Packer C Murray KR Mumuni T Plourde KF Chen M Olumide A Ojengbede O McCarraher DR A Social Media-Based Support Group for Youth Living With HIV in Nigeria (SMART Connections): Randomized Controlled Trial J Med Internet Res 2020 06 02 22 6 e18343 10.2196/18343 32484444 v22i6e18343 PMC7298637 Verbiest MEA Corrigan C Dalhousie S Firestone R Funaki T Goodwin D Grey J Henry A Humphrey G Jull A Vano M Pekepo C Morenga LT Whittaker R Mhurchu CN Using codesign to develop a culturally tailored, behavior change mHealth intervention for indigenous and other priority communities: A case study in New Zealand Transl Behav Med 2019 07 16 9 4 720 736 10.1093/tbm/iby093 30388262 5155648 McConnell MV Shcherbina A Pavlovic A Homburger JR Goldfeder RL Waggot D Cho MK Rosenberger ME Haskell WL Myers J Champagne MA Mignot E Landray M Tarassenko L Harrington RA Yeung AC Ashley EA Feasibility of Obtaining Measures of Lifestyle From a Smartphone App: The MyHeart Counts Cardiovascular Health Study JAMA Cardiol 2017 01 01 2 1 67 76 10.1001/jamacardio.2016.4395 27973671 2592965 Huberman B Adar E Fine L Valuating Privacy IEEE Secur. Privacy Mag 2005 09 3 5 22 25 10.1109/msp.2005.137 Heidel A Hagist C Schlereth C Pricing through health apps generated data-Digital dividend as a game changer: Discrete choice experiment PLoS One 2021 16 7 e0254786 10.1371/journal.pone.0254786 34310618 PONE-D-20-36627 PMC8312968 Vlaev I King D Darzi A Dolan P Changing health behaviors using financial incentives: a review from behavioral economics BMC Public Health 2019 08 07 19 1 1059 10.1186/s12889-019-7407-8 31391010 10.1186/s12889-019-7407-8 PMC6686221 Promberger M Dolan P Marteau TM "Pay them if it works": discrete choice experiments on the acceptability of financial incentives to change health related behaviour Soc Sci Med 2012 12 75 12 2509 14 10.1016/j.socscimed.2012.09.033 23102753 S0277-9536(12)00697-1 PMC3686527 Short K Chadwick J Cannady T Branam D Wharton D Tullier M Thompson D Copeland K Using financial incentives to promote physical activity in American Indian adolescents: A randomized controlled trial PLoS ONE 2018 6 1 13 6 e0198390 10.1371/journal.pone.0198390 29856832 PONE-D-18-02905 PMC5983431 Voyer B ‘Nudging’ behaviours in healthcare: Insights from behavioural economics British Journal of Healthcare Management 2015 03 02 21 3 130 135 10.12968/bjhc.2015.21.3.130 van Roy R Zaman B Why Gamification Fails in Education and How to Make It Successful: Introducing Nine Gamification Heuristics Based on Self-Determination Theory Serious Games and Edutainment Applications 2017 Cham Springer International Publishing 10.1007/978-3-319-51645-5_22 Martin A Caon M Adorni F Andreoni G Ascolese A Atkinson S Bul K Carrion C Castell C Ciociola V Condon L Espallargues M Hanley J Jesuthasan N Lafortuna CL Lang A Prinelli F Puidomenech Puig E Tabozzi SA McKinstry B A Mobile Phone Intervention to Improve Obesity-Related Health Behaviors of Adolescents Across Europe: Iterative Co-Design and Feasibility Study JMIR Mhealth Uhealth 2020 03 02 8 3 e14118 10.2196/14118 32130179 v8i3e14118 PMC7076410 O'Connor S Hanlon P O'Donnell CA Garcia S Glanville J Mair FS Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies BMC Med Inform Decis Mak 2016 09 15 16 1 120 10.1186/s12911-016-0359-3 27630020 10.1186/s12911-016-0359-3 PMC5024516 Glasgow RE Nelson CC Kearney KA Reid R Ritzwoller DP Strecher VJ Couper MP Green B Wildenhaus K Reach, engagement, and retention in an Internet-based weight loss program in a multi-site randomized controlled trial J Med Internet Res 2007 05 09 9 2 e11 10.2196/jmir.9.2.e11 17513282 v9i2e11 PMC1874718 Jakob R Harperink S Rudolf AM Fleisch E Haug S Mair JL Salamanca-Sanabria A Kowatsch T Factors Influencing Adherence to mHealth Apps for Prevention or Management of Noncommunicable Diseases: Systematic Review J Med Internet Res 2022 05 25 24 5 e35371 10.2196/35371 35612886 v24i5e35371 PMC9178451 Denney-Wilson E Laws R Russell CG Ong K Taki S Elliot R Azadi L Lymer S Taylor R Lynch J Crawford D Ball K Askew D Litterbach EK J Campbell K Preventing obesity in infants: the Growing healthy feasibility trial protocol BMJ Open 2015 11 30 5 11 e009258 10.1136/bmjopen-2015-009258 26621519 bmjopen-2015-009258 PMC4679836 Richards D Timulak L O'Brien E Hayes C Vigano N Sharry J Doherty G A randomized controlled trial of an internet-delivered treatment: Its potential as a low-intensity community intervention for adults with symptoms of depression Behav Res Ther 2015 12 75 20 31 10.1016/j.brat.2015.10.005 26523885 S0005-7967(15)30045-0 Richards D Timulak L Doherty G Sharry J Colla A Joyce C Hayes C Internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression: protocol for a randomized controlled trial BMC Psychiatry 2014 05 21 14 147 10.1186/1471-244X-14-147 24886179 1471-244X-14-147 PMC4036670 Bricker JB Mull KE McClure JB Watson NL Heffner JL Improving quit rates of web-delivered interventions for smoking cessation: full-scale randomized trial of WebQuit.org versus Smokefree.gov Addiction 2018 05 113 5 914 923 10.1111/add.14127 29235186 PMC5930021 Edney S Plotnikoff R Vandelanotte C Olds T De Bourdeaudhuij I Ryan J Maher C "Active Team" a social and gamified app-based physical activity intervention: randomised controlled trial study protocol BMC Public Health 2017 11 02 17 1 859 10.1186/s12889-017-4882-7 29096614 10.1186/s12889-017-4882-7 PMC5667488 Ybarra M Goodenow C Rosario M Saewyc E Prescott T An mHealth Intervention for Pregnancy Prevention for LGB Teens: An RCT Pediatrics 2021 03 147 3 e2020013607 10.1542/peds.2020-013607 33568491 peds.2020-013607 PMC7924142 Looyestyn J Kernot J Boshoff K Maher C A Web-Based, Social Networking Beginners' Running Intervention for Adults Aged 18 to 50 Years Delivered via a Facebook Group: Randomized Controlled Trial J Med Internet Res 2018 02 26 20 2 e67 10.2196/jmir.7862 29483065 v20i2e67 PMC5847819 Heber E Ebert DD Lehr D Nobis S Berking M Riper H Efficacy and cost-effectiveness of a web-based and mobile stress-management intervention for employees: design of a randomized controlled trial BMC Public Health 2013 07 15 13 655 10.1186/1471-2458-13-655 23855376 1471-2458-13-655 PMC3717042 Dalton C Carlson S Butler M Cassano D Clarke S Fejsa J Durrheim D Insights From Flutracking: Thirteen Tips to Growing a Web-Based Participatory Surveillance System JMIR Public Health Surveill 2017 08 17 3 3 e48 10.2196/publichealth.7333 28818817 v3i3e48 PMC5579323 De Brouwer W Patel CJ Manrai AK Rodriguez-Chavez IR Shah NR Empowering clinical research in a decentralized world NPJ Digit Med 2021 07 01 4 1 102 10.1038/s41746-021-00473-w 34211085 10.1038/s41746-021-00473-w PMC8249659 Apostolaros M Babaian D Corneli A Forrest A Hamre G Hewett J Podolsky L Popat V Randall P Legal, Regulatory, and Practical Issues to Consider When Adopting Decentralized Clinical Trials: Recommendations From the Clinical Trials Transformation Initiative Ther Innov Regul Sci 2020 07 54 4 779 787 10.1007/s43441-019-00006-4 32557302 10.1007/s43441-019-00006-4 PMC7362888 Field N Cohen T Struelens MJ Palm D Cookson B Glynn JR Gallo V Ramsay M Sonnenberg P Maccannell D Charlett A Egger M Green J Vineis P Abubakar I Strengthening the Reporting of Molecular Epidemiology for Infectious Diseases (STROME-ID): an extension of the STROBE statement Lancet Infect Dis 2014 04 14 4 341 52 10.1016/S1473-3099(13)70324-4 24631223 S1473-3099(13)70324-4 Agarwal S LeFevre AE Lee J L'Engle K Mehl G Sinha C Labrique A WHO mHealth Technical Evidence Review Group Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist BMJ 2016 03 17 352 i1174 10.1136/bmj.i1174 26988021