We have conducted a narrative literature review of articles published up to April 4, 2019, of the validation and monitoring frameworks of BITs. The main databases that we searched were MEDLINE, PsycINFO, Web of Science, Science Direct, and the Journal of Medical Internet Research (JMIR) database. They have been chosen as reference databases in biomedicine and psychology and comprise the largest general scientific database [24]. The search keywords were as follows: (“model” OR “framework” OR “process” OR “evaluation process” OR “validation process”) AND (“behavior” OR “behavior change” OR “behavioral intervention”) AND (“digital health” OR “ehealth” OR “mhealth” OR “connected health” OR “medical app” OR “smartphone” OR “iphone” OR “email” OR “text message” OR “SMS” OR “mobile app” OR “smartphone app” OR “connected health” OR “wearable”). New articles were extracted based on analyzed articles.

The frameworks were listed in chronological order. Items for each framework were as follows: name of the framework—if none was given, a phrase describing it in the original article was written within quotation marks; publication year; name of the first author, called the creator here; country of the creator; organization having funded the creator; health focus or lack of health focus; the target group (ie, population for whom the framework had been initially designed, for example, researchers, health professionals, or software designers); and its design. For the latter, we noted the involvement of the final users (ie, early stage or systematic) and the chain of the development stages (ie, linear, iterative, evolutive, and/or concurrent) [25]. A linear process executes the different stages in a sequential order. An iterative process combines one or more stages before chaining. An evolutive process executes them in a circular pattern in order to obtain, with each revolution, a more mature version of the product. Finally, a parallel process executes one stage or more in a concurrent way. If a framework had evolved through time and had been described by a new publication, the latter was also written in the synthesis table but its creator, as well as the creator’s country and funding organization, remained the original one.

The frameworks were categorized according to the accepted procedure of translational research [26,27]. It may be described as a continuum of steps. It starts with a prototyping step, which has a background in engineering, followed by a mechanisms step based on a theoretical approach to health behavior change. Concept follows, which is a proof-of-concept step regarding health impact based on an exploratory intervention trial. This is followed by a demonstration step, evidence of health efficacy and effectiveness, which is based on a confirmatory interventional trial. Finally, a surveillance step is implemented for market use based on implementation and dissemination studies.

The frameworks follow three general paradigms: (1) the biomedical paradigm, with its essential phase of clinical research to identify the benefits and risks; (2) the engineering paradigm, with its essential phase to improve the device; and (3) the behavioral paradigm, with its essential phase to evaluate the impact on health behaviors.

The literature review identified 46 frameworks, besides the five-phase drug development model, that met the research criteria (see Table 1).

The categorization of frameworks, according to their coverage of the continuum of steps, is shown in Figure 1.

The Waterfall model is the only one that covers only the prototyping step [28]. Other frameworks cover the prototyping and surveillance steps by integrating the final user [30,33-36,39,40,46,48,49,68].

The behavior change wheel and the pragmatic framework for developing just-in-time adaptive interventions (JITAIs) are focused on the mechanisms step [51,63]. The BIT model, the chronic disease mHealth app intervention design framework, and the Network for the Improvement of Addiction Treatment (NIATx) model cover the prototyping and mechanisms steps by associating the conception of both numeric and behavioral interventions [12,17,65].

The Explore Values, Operationalize and Learn, and eValuate Efficacy (EVOLVE) and MOST frameworks cover the mechanisms, concept, and evidence steps. A total of 9 frameworks out of 46 (20%) are extended over the mechanisms, concept, evidence, and surveillance steps [29,31,32,37,44,47,49,58,62]. Finally, the whole continuum of steps is covered by 12 out of 46 (26%) frameworks [17,41,43,52,56,57,61,64,66,70,71,74].

Figure 1 illustrates the categorization of frameworks based on their paradigms. Most belonged to one paradigm (ie, biomedical, engineering, or behavioral), while few referred to two or three paradigms.

Out of 46 frameworks, 12 (26%) fell within the paradigm of engineering [28,30,33-36,39,40,46,48,49,68] and 3 (7%) were based on the behavioral paradigm [38,51,63]. A total of 9 (20%) frameworks were based on these two latter paradigms [12,55,59,60,65,67,70-72]. A total of 5 (11%) frameworks were based on the biomedical paradigm, which draws on the clinical trial, specifically the randomized controlled trial [28,31,32,44,52]. A total of 4 others (9%) also incorporated a theoretical approach of behavior change [37,47,54,58]. The framework for evaluating emergent eHealth resources was based on the biomedical and engineering paradigms [43]. A total of 12 (26%) frameworks mixed three paradigms [17,41,42,56,57,61-63,67,69,73,74].

Our narrative review of the literature showed the absence of a unique and consensual validation and surveillance framework for health behavior intervention technology. This conclusion is in keeping with the statement of Bradway et al about mHealth assessment: “too many initiatives, too few answers” [75]. To date, 46 frameworks coexist, besides the five-phase drug development model, without a predominance of, or a convergence toward, one of them. Their number has increased exponentially over time since the 1970s. The United States and the United Kingdom are the main countries that have proposed models, probably due to their university research productivity and new-technology industry leadership [76]. They are motivated, on the one hand, by the plethoric and easily available offer of digital tools and networks and, on the other hand, by the failure to prevent unhealthy diet, addiction, and physical inactivity [2].

Beyond the quantitative increase of these frameworks, this review underlines an increased heterogeneity. One surprising finding is that the benefit-to-risk ratio is not a main goal. Contrary to drug claims regarding societal recognition and reimbursement for their impact on health outcomes [77], showing evidence of safety and efficacy is paradoxically not a priority for BITs. This fact can explain why researchers have noted that medical mobile phone apps are lightly used by patients in the medium term, have little involvement from health professionals, lack characterization of their content, and have a low integration of behavior change theories [78]. The same problem can be found for connected health devices. The lack of evidence of their efficacy leads to their nonprescription by general practitioners, who picture them mainly as a supplementary organizational constraint [79]. Most frameworks are inspired by the paradigms of engineering and behavioral sciences. Engineering is focused on technicity (ie, release criteria, ergonomics, user experience, and data security), which leads to uncertainties about these technologies’ health contributions and turns them into entertainment products or gadgets [80]. The behavioral sciences bring an understanding of behavior change mechanisms and skills techniques without determining and comparing their health impacts [81,82]. This explains how translational research is unusual in this area compared to that for drugs [83]. To date, for instance, there is no validation model for psychotherapies [84].

Beyond the quantitative increase and diversification of frameworks, we have noted the emergence of common principles. The first one is an assumed superiority with respect to health and health behavior change of some BITs compared to others. This hypothesis aims to address the plethoric and exponential offer of digital solutions sustained by marketing. This growing need to know the efficacy of each BIT and to compare them to other solutions to change health behaviors comes from both health professionals, who seek to recommend or even prescribe evidence-based solutions, and health users, who seek solutions beyond marketing and/or users’ experiences from social networks.

The second is the requirement of user involvement during the beginning of BIT development and after market access. This principle exists in every recent framework based on the behavioral science, engineering, or biomedical paradigms. It requires a coconstruction with the user and considers his or her experience. It involves the end user through a user-centered design from engineering [85,86]. It requires a patient-centered approach to the provision of personalized care in the biomedical area [87]. The patient’s preferences are considered and may allow improved chances of stable and efficient use.

The third is the ambition to evaluate BITs beyond simple satisfaction. This is more or less oriented by manufacturers to aim at effectiveness in medicine becoming more and more predictive, preventive, personalized, and participative; this is called 4P medicine [88,89].

The fourth is the shortening of the delay for upstream validation, until its complete suppression in some engineering frameworks, and a multiplication of the downstream surveillance methods, such as implementation studies, experience feedback (eg, to a learned society or health authority), and big data analyses combining multimodal data. The shortening of the upstream period is linked, on the one hand, to a supposedly low dangerousness of these health solutions and, on the other hand, to the short life cycle of the technologies involved [90]. The development time of a drug, from the lab bench to its marketing authorization, is between 17 and 24 years [91-93]. It is incompatible with digital innovation. Digital industrialists assume that hundreds of millions of users will allow for the evaluation of BITs’ effectiveness on a large scale thanks to big data analyses [94,95].

The fifth is the introduction of hybrid frameworks integrating the development and updating processes from engineering, behavior change theories, techniques from behavioral sciences, and the rigorous approach of validation from the biomedical area (eg, the FDA and the Environmental Management Association). This hybridization contributes to the creation of interventions standing between medical devices and products in the category of general commodities. The scientific process to develop and validate them, the rules for marketing authorization, and surveillance have not yet been defined for a country or a continent and are still under construction. Many digital industrialists wish to avoid the validation process for medical devices in view of its constraints and costs; at the same time, they aim to indicate to health professionals and users the health impact of their solutions. They are encouraged to address this by mutual insurances, other insurances, and banks.

We stress two extreme positions for the studied frameworks that may become problematic for the development of BITs in the health sector. The first is that of permanent technological innovation. It is embodied in the Agile software management framework. Flexibility, based on user demand, dictates each evolution. Health is a market like any other. If this approach was to win in this area, for example, to make substantial savings in clinical research, no counterpart could be asked for, concerning health benefits. Big data analyses will not guarantee a relevant comparability between BITs [96]. The second problem at the opposite extreme is that marketing authorization of the BIT must be conditional on the completion of a well-designed randomized controlled trial. This process has been a part of the success of drugs in the last century. These trials are long, costly, and debatable in terms of methodology for BITs, if only for the choice of control group [97].

To identify every validation and surveillance framework of BITs is challenging. This area is new, it is based on a number of different paradigms, and develops in an uncontrolled way from one country to another and from one industry to another. One of the identified difficulties within these publications is the lack of common terminology [98,99]. The originality of our review is the spanning of the three approaches (ie, engineering, behavioral sciences, biomedical sciences). Although they are often opposed, the concern here was to note the borrowings and contributions that they could mutually bring through transversal thinking.

Validating a BIT for health has the following specific challenges: achieve an efficient and quick development; understand and promote long-term adherence; be supported by behavior change theories and techniques; evaluate effectiveness and cost-effectiveness; and ensure rigorous management in terms of regulation, ethics, and information [93]. Patient expectations are relayed through the media and through the arrival on the health market of digital players proposing short life-cycle products due to rapid obsolescence of technology; this justifies the need for a consensual validation and surveillance framework for nonpharmacological interventions. This framework will necessarily differ from that used for drugs. Our review has identified 46 frameworks, none of which dominates for BITs. Three paths are opened: (1) the anarchic continuous development of new competing frameworks that prevents any convergence in a standardized validation and surveillance framework, with its consequential recognition by health authorities; (2) the movement toward the type of framework for drug evaluation centered on establishing its effectiveness before marketing authorization to guarantee its safety for users, which is heavy and costly; and (3) the implementation of a framework reliant on big data analysis, based on a posteriori research and an autoregulation of a market; however, that does not address the safety risk for the health user, as the market will not regulate safety or efficacy issues. This article calls for a minimal upstream clinical phase and an increased surveillance of BITs to address the risk of semantic amalgams, inappropriate prescriptions, and induced misuses [16,100]. A BIT cannot be a simple tool but must be a complex strategy integrated in a given environment [101]. A BIT must make sense by virtue of its interaction with the context, building a system, and must be evaluated as such [100].