Two authors (SMS and YL) with the assistance of a medical librarian (RSH) devised systematic strategies to search the Ovid MEDLINE, Ovid EMBASE, PubMed, Cochrane Library, Scopus, and PsychInfo databases for all relevant literature published through December 2016. Searches were limited articles written in the English language and conducted in humans. Database search strategies included the use of controlled vocabulary (eg, Medical Subject Headings and Emtree) and keywords to identify studies addressing PA or diet in conjunction with feedback. Keywords included physical activity, exercise, diet, eating, intervention, and feedback. Additionally, the bibliographies of topically relevant review papers and all included studies were examined to identify any additional studies.

Eligible studies included (1) Just-in-time feedback as an intervention component and (2) Targeted behavior changes that included PA, sedentary behavior, or dietary intake. Just-in-time feedback was defined on a case-by-case basis as any feedback that focused on participants’ daily PA, sedentary behavior, or dietary intake and that was provided within 1 min to 1 day of assessing current performance, as appropriate to each intervention or behavior change goal. Studies with multiple feedback components were included, but at least one type of delivered feedback must have met this definition of just-in-time feedback.

Studies were excluded if (1) the intervention-targeted behaviors were off-topic (eg, studies of clinical education, personnel, management, medication adherence, blood glucose self-monitoring, and symptom management); (2) no intervention outcome results were reported (eg, protocol papers); (3) the time frame for providing feedback was greater than 1 day (eg, weekly performance summaries); (4) no or inadequate feedback was provided (eg, studies of performance tracking without evaluation); or (5) they reported the secondary outcomes of an included intervention.

Data were extracted into a structured coding form according to Preferred Reporting of Systematic Reviews and Meta-Analyses (PRISMA) guidelines [18] and the Cochrane Handbook for Systematic Reviews of Interventions [19]. Five reviewers (SMS, YL, MCR, MEH, and TB) extracted characteristics and outcomes from all studies using a standardized data extraction form. The following information was extracted: (1) general study characteristics (ie, country of study, study type, participant population, participant demographics, and study sample size); (2) intervention characteristics (ie, intervention name, study design, intervention duration, behavioral theories used, and intervention goals); (3) just-in-time feedback characteristics (eg, content, delivery frequency, and delivery mode); and (4) intervention results (ie, within-group changes and between-group comparisons in targeted behaviors and weight outcomes). A copy of the data extraction form is provided as Multimedia Appendix 1. A comparison of the extracted data across reviewers was conducted (SMS and YL). Differences in the extracted data were resolved by a discussion between the expert reviewers (SMS and YL) to complete the dataset.

Assessing each study for risk of bias was performed using the 2010 version of the Quality Assessment Tool for Quantitative Studies [20]. An overall quality score for each study was assigned based on the ratings for six domains: (1) selection bias, (2) study design, (3) confounders, (4) blinding, (5) data collection methods, and (6) withdrawals and dropouts. The information extraction and quality assessment for each study was performed independently by two reviewers (YL and MCR). Discrepancies between reviewers’ ratings were resolved through discussions that led to a consensus (YL, MCR, and SMS).

A meta-analysis was not possible owing to substantial heterogeneity in study design, study quality, intervention type, and outcome measures, as well as a lack of studies that explicitly tested the efficacy of using feedback as a BCT. Furthermore, the primary and secondary outcomes of the studies varied widely. We limited our summary of results to primary and secondary study outcomes that were specific to changes in PA, dietary intake, or body weight or body composition. Consistent with the design of a previous review [6], two key criteria were used to determine whether a study explicitly tested feedback: (1) the study had at least one treatment group that received feedback that was not confounded with other manipulations (not matched in the control group) and (2) the study included at least one control group or quasi-control group that received no feedback. Data were synthesized narratively rather than quantitatively.

The literature search yielded 4239 studies, of which 909 were duplicates, leaving 3330 articles to be screened for eligibility. A total of 3083 articles were excluded upon title or abstract screening because they were unrelated to diet or PA, had no reported outcomes, were nonintervention studies, or had ineligible feedback features. Thus, 246 full-text articles were assessed for eligibility. After 215 articles that did not meet the inclusion criteria were excluded, 31 studies with a total of 6623 participants were included in the review (see PRISMA diagram, Figure 1).

The studies’ characteristics are summarized in Multimedia Appendix 2. Studies varied by the behavior about which feedback was provided and by their sample size, population, design, and duration. Of the 31 studies, 3 focused on diet- or nutrition-related behavior only [21-23], 15 focused on PA or sedentary behavior only [24-38], and 13 focused on both diet and PA [39-51]. The median number of study participants was 83 (range=10-1488). Studies were conducted in the United States, the United Kingdom, the Netherlands, Australia, Belgium, Denmark, Hong Kong, Ireland, Japan, Portugal, and South Korea and included adults (≥18 years), children (<18 years), or young adults (17-26 years). The participants’ weight statuses were not consistently reported, but at least 11 studies enrolled only overweight or obese individuals [21,22,24,25,39,40,43,47,49-51]. A total of 28 studies were randomized controlled trials (level of evidence I) [21-24,26,27,29-41,43-50], 5 of which used clustered randomization [23,31,32,35,36] (level of evidence I). The 4 remaining studies used within-subjects single-arm [28,42,51] or counterbalanced designs [25] (level of evidence II). Study duration ranged from 2 weeks to 24 months. Four studies [31,32,35,41] had postintervention follow-up periods that ranged from 4 weeks to 6 months.

The 31 studies’ risks of bias are summarized in Multimedia Appendix 3. Using the current Quality Assessment Tool for Quantitative Studies [20], we determined that 18 studies had a moderate global rating, 9 had a weak global rating, and 4 had a strong global rating. All but 4 studies received a strong study design rating for being randomized controlled trials. A total of 25 studies received strong scores for controlling for potentially confounding variables, 23 studies used data collection measures with demonstrated reliability and validity, and 19 studies had retention rates of ≥80% across conditions. The risk of selection bias posed the greatest threat to validity; 22 studies received a weak score in this domain. Blinding was rated as weak in 4 studies; however, the assessment tool we used may have underestimated this bias [52]. Most studies did not describe blinding procedures for research staff or participants.

The 31 studies varied in the content, frequency, timing, and delivery of feedback, as well as in the methods used to monitor targeted behaviors and the theoretical foundations that guided the feedback content. Multimedia Appendix 4 provides a full description of the feedback features the studies used and a summary of the key features described below.

The studies’ outcomes are described in Multimedia Appendix 5. We determined that 9 studies explicitly tested the use of feedback to motivate behavior change or significantly modulate body weight or body composition or glycated hemoglobin. Of the 9 studies that tested feedback efficacy, 1 was in the area of diet- or nutrition-related behavior [21], 5 focused on PA or sedentary behavior [26,27,30,34,37], and 3 focused on diet and PA or sedentary behavior [39,45,49]. Most of the remaining studies tested a comprehensive intervention in which feedback was implemented as one of the multiple behavior change strategies. Of the 9 studies that tested the efficacy of feedback, 4 had significant findings [21,30,37,45] and 5 did not [26,27,34,39,49].

Regarding feedback content, 3 of the 4 studies with significant findings [21,30,45] and 3 of the 5 studies without significant findings [27,34,39] used both goal- or target-oriented feedback and actionable feedback. In addition, feedback was provided continuously in 3 of the 4 studies with significant findings [21,30,37] and in only 1 of the 5 studies without significant findings [49]. Concerning feedback prompting, 3 of the 4 studies with significant findings used objectively collected data and passive feedback delivery methods [21,30,37], and the fourth [45] used both passive and user-initiated assessment and feedback methods.

On the basis of the results of our review, we developed a practical framework (Figure 2) that highlights key factors to be considered when developing just-in-time feedback for technology-enhanced diet and PA interventions. We suggest that behavioral objectives (goals or targets) serve as the guiding context for just-in-time feedback and that the selected method of behavioral assessment is the foundation that determines the level of interaction between a user or participant and an external agent or researcher. We propose three characteristics of feedback to take into account: timeliness, personalization, and action orientation.

Our comprehensive literature search yielded 31 studies that met our eligibility criteria. Most of these studies provided feedback that was just-in-time (100%, 31/31), goal-oriented (75%, 23/31), and personalized (97%, 30/31); however, only 5 studies (15%, 5/31) provided actionable feedback. Interventions with significant effects featured continuously available and personalized feedback that was actionable or goal-oriented and used objectively assessed data and passively initiated feedback delivery methods. On the basis of these findings, we have proposed a framework for designing just-in-time feedback that incorporates three key content characteristics (timeliness, personalization, and action orientation) relative to a known behavioral goal or target and highlights important considerations regarding the quality and frequency of the behavioral assessment from which just-in-time feedback is derived.

To our knowledge, our review is unique in that its primary goal was to examine the use of just-in-time feedback in diet and PA interventions. Only one other similar systematic review has been conducted. In that review [17], the authors concluded that the use of personalized feedback was an effective BCT in weight-loss interventions after demonstrating that individuals receiving Internet-delivered personalized feedback lost on average 2 kg more than those receiving no personalized feedback; however, the authors did not analyze the effectiveness of other feedback characteristics. In another review, Michie and colleagues [56] demonstrated through a meta-regression analysis that healthy eating and PA interventions that implemented behavioral monitoring plus one BCT (eg, action planning, prompting specific goal setting, providing feedback on performance, and prompting review of behavioral goals) were more effective than those that did not. However, the authors acknowledged the need to experimentally test the most effective combination of BCTs [56]. Furthermore, only one other known model for developing feedback exists. Hysong and colleagues [57] examined the use of feedback for improving clinical practice guideline adherence and developed the model of actionable feedback. The model posits that an optimal effect on clinical performance can be achieved by providing feedback that is timely, individualized, nonpunitive, and based on customizable performance data. Action was an implied outcome of providing optimal feedback in the model. These previous studies highlight some potentially effective components of feedback that can motivate goal-directed behavior change and that overlap with those put forth in this review.

Another key finding of our review was that only a few of the reviewed studies provided goal-oriented feedback that was actionable. Of the 31 studies included in the review, 23 incorporated behavioral goals; however, only 5 [21,25,30,39,45] gave actionable suggestions on when, where, and how to achieve those goals. In theory, intention precedes action [58]; however, research shows that the intention alone does not often result in actual behavior change [59-61]. Action planning or intention implementation is one strategy to help people transform their intention into action [54]. Traditionally, individuals conceive action plans before acting as their commitment to perform behaviors when opportunities arise [54] and can reevaluate these plans weekly [62]. Such plans often describe where, when, or how frequently to perform the behavior (eg, I intend go to the gym 3 days per week) [55]. Individuals may or may not receive feedback regarding their action plans. In this study actionable or actionable feedback refers more specifically to information in the form of suggestions an external agent provides to an individual about where and when to perform the behavior and instructions on how to perform the behavior. Examples of actionable feedback in the reviewed studies included multiple, daily behavioral prompts to perform PA breaks of specific durations [25], daily booster messages with exercise prescriptions to achieve the current day’s step goal [30], and just-in-time dietary recommendations (eg, Be aware of high fat snacks tonight) [39]. Our definition of actionable feedback is consistent with Michie and colleagues’ Coventry, Aberdeen, and London-Refined (CALO-RE) taxonomy of BCTs [55]. It is also consistent with the concept of providing supportive information, advice, and feedback at critical moments in just-in-time adaptive interventions [5,11]. This review demonstrates that the provision of actionable feedback in diet and PA interventions is an underutilized behavior change strategy. Relatedly, reviews of seven wearable activity trackers and 40 top-rated diet and PA mobile phone apps found that the integration of BCTs related to action planning and providing instructions consistent with CALO-RE taxonomy was not uncommon [63,64]. However, only two activity trackers helped users identify where and when to perform PA. Given the increased bidirectional communication capabilities offered by the technology being used to facilitate behavioral interventions, actionable feedback as defined here will likely be more frequently incorporated into interventions as a supportive behavior change strategy. One example of such an intervention is the MyBehavior mobile app. MyBehavior is based on the Fogg behavioral model that applies theoretical principles to technology design by creating tools to prompt low-effort actions that can be triggered even when motivation is low [65]. MyBehavior was designed to generate personalized, actionable insights on when, where, and how to achieve the set goals [66]. The behavioral outcomes from the MyBehavior trial have not yet been published. Future research to determine the efficacy of actionable versus nonactionable feedback as a behavior change strategy will be needed.

Our review is strengthened by its focus on key theory-based characteristics of feedback delivered as behavior change interventions. We focused on only diet and PA interventions rather than looking more broadly across additional health behaviors. We did this in part because it is unclear how generalizable our findings might be to other behaviors or health-related outcomes. In addition, studies eligible for inclusion could have included multiple types of feedback, but at least one form of feedback had to meet our definition of just-in-time feedback. We believe this approach strengthens this review by enabling us to make conclusions that facilitate the progress of intervention science into a future in which feedback can be generated and delivered just in time, thereby preparing researchers for continued advancements in technology. Finally, our synthesis of the available data enabled us to develop a framework for designing just-in-time feedback for health behavior change interventions.

Despite these strengths, we were unable to conduct a meta-analysis primarily because of the variability in targeted behavior and study outcomes. Additionally, because it was not clear whether the included studies monitored the delivery, receipt or viewing, or comprehension of the provided feedback, we were not able to conclusively determine the efficacy of using feedback or which feedback feature(s) might be more effective than others. However, we found that feedback was continuously available, goal-oriented, or actionable in 3 of the 4 studies with significant intervention effects. In addition, the sample sizes, intervention durations, and interventions outcomes of studies with significant findings ranged widely. Interventions of longer duration could have been more likely to have significant findings. However, the duration of studies with significant findings was generally shorter than studies with nonsignificant findings (4 weeks to 12 months vs 2 weeks to 24 months). Excluding studies for providing feedback more than 24 hours after a person performed the target behavior limited the number of eligible studies; however, we believe that it was consistent with the advancement of body sensor technology and therefore important to the context of the review. Another important limitation to consider is that most of the included studies did not recruit participants from a representative, diverse population, thus limiting the generalizability of the findings. Finally, the cost of wearable body sensors and wireless devices could be a potential limitation for scaling up the technology-based interventions.

Before this study, few reviews had critically examined the use of feedback in diet and PA interventions. As advancements in technology continue to improve bidirectional communication between investigators and their participants, optimizing feedback messages will be key to future interventions. The systematic review and the framework we propose represent a foundation for designing feedback messages for future just-in-time diet- or PA-based interventions. Investigators may use the framework to ensure feedback developed for their interventions contain content that is theoretically and empirically supported to have a positive effect on behavior change. However, it is unclear from this review how many of the proposed components are needed to effectively motivate behavior change. Empirical research will be needed to determine the optimal combination of feedback components.