Public health would substantially improve if a large number of people adopted a healthy lifestyle, encompassing among others, ample physical activity, and healthy diets [1]. To initiate or coach such change, embodied conversational agents (ECAs) can be a valuable tool. ECAs can be defined as “more or less autonomous and intelligent software entities with an embodiment used to communicate with the user” [2]. Examples include those given in Figure 1; From left to right: Laura [3], Gabby [4], and an anonymous octopus [5]. An example of an early ECA is Laura [3]. Laura interacts daily with users to motivate them to be more physically active. She uses several relational behaviors, such as social dialogue, feedback, humor, facial expressions, and body language. Through these behaviors, users establish and maintain a meaningful relationship [3]. What makes ECAs unique for coaching people with respect to their health is this capacity of establishing and maintaining an empathic relationship [3], a relationship characteristic proven to be the most crucial factor for successful lifestyle coaching [6]. In addition, ECAs are available 24×7. Consequently, they can offer empathic support when it matters most: immediately before or after specific behavior, which maximizes impact [7].

Despite the promising role ECAs can play in coaching people for a healthy lifestyle, literature that discusses how to develop them and demonstrates their effectiveness is scarce. A review by Provoost et al [8] provides some insight into the developmental processes and evidence base of ECAs for coaching people with mental disorders. They suggest that the more rigorous studies put little emphasis on design and that evidence on clinical effectiveness remained sparse [8]. In the educational context, Johnson and Lester [9] state that there is a significant body of experience and research findings related to pedagogical agents. However, similar to the health context, many questions remain about when pedagogical agents are most effective and how they should be designed and used to maximize effectiveness. Literature on development and effectiveness is essential to create ECAs that can have a high level of impact and uptake, a problem with which electronic health (eHealth) interventions constantly struggle [10]. The cause for this low impact and uptake is often attributed to a misfit among technological, human, and contextual factors during development [11,12]. Different authors have therefore recommended to apply a human-centered and stakeholder-inclusive design approach, as well as to incorporate persuasive design features in the technology [11,13,14].

This scoping review identifies the current developmental practices of ECAs for coaching people in a healthy lifestyle, and it provides an overview of their efficacy and use-related outcomes. For researchers, this review provides an overview of the potential ECAs have to change people’s lifestyle and identifies the most urgent research questions related to this domain. For practitioners, the review will lead to actionable advice for devising a development trajectory for this type of ECAs.

The Arksey and O’Malley framework for scoping reviews [15] was adopted, which distinguishes 5 different stages: (1) identifying the research question, (2) identifying relevant studies, (3) selecting studies, (4) charting the data, and (5) collating, summarizing, and reporting the results.

Inclusion criteria were implemented by selecting different options and limits during the search (see Multimedia Appendix 1). The results of the search query were uploaded into the EndNote reference manager (Thomson Reuters) and independently assessed by 2 reviewers (LK and SS) to decide on their inclusion based on title, abstract, and full text. Conflicts between the 2 reviewers were identified after each step, independently; arguments were formulated per study and then discussed and resolved. This process was documented in the logbook. To find relevant studies that describe an intervention with an ECA in the healthy lifestyle domain, the following exclusion criteria were applied: (1) there is no report on primary data, (2) there is no intervention, (3) the intervention does not include an ECA (a “more or less autonomous and intelligent software entities with an embodiment used to communicate with the user”) [2], and (4) the ECA is not used in a lifestyle health behavior context (eg, tobacco use, physical (in)activity, alcohol consumption, and diet) [4].

Figure 2 charts the screening and selection process. In total, 1789 unique studies were identified in the database search. Title and abstract screening resulted in the exclusion of 1754 studies. The remaining 35 studies were screened in full. Of those, 19 studies were excluded as the studies were not an intervention or did not include an ECA. This resulted in a total of 16 studies. One of these studies [4] described both a rehospitalization and a physical activity trial. As the first is not a lifestyle behavior, only the second trial was included in the analysis. A total of 4 more studies were found through snowballing [17-20]. This resulted in a total of 20 studies that were included in this review (see Multimedia Appendix 3 for a complete overview of the study characteristics).

The first studies were published in 2005 [3,17,21]. All the studies were either performed in the United States [3,4,17-19,21-31] or in the Netherlands [5,20,32,33]. Of all the studies performed in the United States, except for 1 study [26], TW Bickmore was listed as the author. A total of 13 studies were in the pilot phase [3,4,17-19,21,24-26,28,30-32], 1 study was in the development phase [22], and 6 studies were in in the evaluation phase [20,23,27,29,31,32]. Thus, none of the studies described the implementation or had actually implemented their ECA in practice. One ECA was used in a community setting and could be accessed via a computer kiosk [29]. All other ECAs were used at home and could be accessed via a website [20,24,26,28,30-32], or software installed on a PC [3,17,19,21-23,25], tablet [4,18,27], or mobile phone [33]. Only 1 ECA was part of an overarching platform, accessible via a website and an Android app [5]. Most studies targeted physical activity [3-5,17-23,25,27,29-33]. Other lifestyle behaviors were nutrition [5,20,25,30], mindfulness [26,30], preconception care [24,28], stress [30], blood glucose monitoring [5], and sun protection [31]. Moreover, one specific study targeted healthy lifestyles among diabetes patients. Patients may differ in their needs for lifestyle support compared with healthy individuals. The diversity in focus and target groups limits the comparability among the studies, and future research could help expand the evidence base for specific ECAs. Study designs varied from a randomized controlled trial (RCT) [3,4,17,19-23,25-30,32,33] to a pretest-posttest design, either with [31] or without control a control group [5,18,30]. Sample size ranged from 9 to 958 participants (median=60.5). Study duration lasted from 4 weeks to 36 months (median=8 weeks).

Across the studies, 9 different names were used to describe an ECA, although the definitions were rather similar. A total of 6 studies used the name embodied conversational agent [3,4,19,26,27,30], whereas the other studies used different names: relational agent [3,17,21,22,31], virtual coach [5,23], virtual exercise coach [18], virtual avatar [32], virtual patient advocate [24], conversational agent [28], animated conversational agent [25], virtual advisor [29], personal digital coach [33], and persuasive computer assistant [20]. A total of 6 studies did not provide a definition for an ECA [5,17,20,23,25,32]. All other studies referred to earlier with TW Bickmore listed as the author used variations of “an interactive, animated computer character that simulates face-to-face counseling” [5].

This scoping review charted the design and evaluation field of ECAs for coaching people in a healthy lifestyle. In total, 20 relevant studies were identified and analyzed. One could argue that the lack of diversity in research teams limits the external validity of the scoping review. However, although the work in this field is dominated by 1 research group, a careful comparison between research groups showed no differences in design and evaluation processes, as well as in outcomes (see Multimedia Appendix 3). We therefore conclude that the developmental processes described in this review are a realistic reflection of the field. Regarding the design, we found that studies often applied multiple theories or principles, but they did not report on their interpretation and application. Human-centered and stakeholder-inclusive design approaches tended to be unused. Regarding the evaluation, a combination of efficacy and use-related outcomes was assessed, usually in an RCT. However, rather than evaluating specific components, the intervention was evaluated as a whole. Overall, the studies included suggest that ECAs for coaching people in a healthy lifestyle can make an intervention more engaging, although evidence on their effectiveness remains inconclusive.

Myriad theories and therapy-derived principles were applied for creating ECAs’ content and communication strategy. As it is difficult to determine what theory or principle best fits a specific context and as it is reasonable to assume that different contexts require the use of different theories and principles, we do not consider this diversity a problematic issue. However, what we do see as problematic is the lack of detail with which the incorporation of these theories and principles into functional or content design of an ECA is reported. If how exactly an ECA works remains unclear, it will be difficult to learn from others’ efforts or interpret the outcomes of evaluations performed with an ECA. This prevents knowledge accumulation about ECAs in general, as well as specific knowledge accumulation about which theories and principles are most appropriate in which contexts. A similar conclusion can be drawn with respect to the design process of ECAs. The design of an ECA can have a major effect on both impact and uptake. On the basis of empirical results of different studies on the appearance of ECAs, Baylor concludes that different appearances lead to different outcomes in terms of motivation and behavior change [36]. Unfortunately, reporting on the design activities and their results is generally incomplete or missing, thereby limiting the options for replication and learning from others’ work. It is therefore recommended that future ECA work should not only present results on the efficacy of the ECA but also on the process leading to the design and content of the ECA.

With respect to the evaluation of ECAs for coaching people in a healthy lifestyle, we made a distinction between the results in ECAs’ efficacy and use-related parameters. ECA outcome efficacy shows a nonconclusive picture, operationalized as, for example, physical activity measured by an activity monitor, knowledge about mindfulness meditation as assessed via a survey, or diabetes-related emotional distress. About half of the evaluation outcomes show a significantly positive result for using an ECA, whereas the other half of the outcomes do not provide positive evidence. With regard to use-related outcomes, the evaluations do show a positive picture, where the majority of the studies indicate that the use of an ECA leads to higher ratings of usability or a higher degree of use. With regard to the efficacy-related outcomes, motivation to change had successfully improved in a majority of the studies, whereas health behavior and health literacy had not. On the basis of the existent evaluations, we can therefore state that ECAs do not necessarily lead to improved health outcomes; however, the intervention will at least be more engaging. This is in accordance with Provoost et al, based on their review of ECAs in clinical psychology and their evidence base [8].

We found that end users are normally not involved with the visual design and content of the ECA. Rather, the ECAs were designed by professionals behind a desk. This practice contradicts human-centered or collaborative design approaches that are assumed to lead to technology appealing to and fitting the perspectives of the end users [37]. This consequently maximizes the chance of successful uptake of the technology [10]. In the literature, several practical approaches for human-centered design for eHealth are provided, such as the Centre for eHealth and Wellbeing roadmap [11] or Integrate, Design, Assess, and Share [38], as well as a rich collection of case studies in which these approaches have been used [39,40]. The field of developing and evaluating ECAs for eHealth would highly benefit from the reporting of similar case studies in diverse contexts.

We found that the evidence for using ECAs for coaching people in a healthy lifestyle remains inconclusive and that it is unclear which (combination of) components caused a (lack of) behavior change. However, this problem is neither new nor exclusive to the field of ECAs; this so-called black box phenomenon has been acknowledged for eHealth interventions in general [32,41]. Rather than evaluating an eHealth technology or ECAs for health purposes as a whole, an evaluation should focus on gaining insight into the effectiveness of the technology’s or ECA’s main or constituent components. A more fine-grained evaluation can be achieved by means of a factorial design, as this allows researchers to deliver specific intervention components to different groups of users [42]. Another strategy is to collect log files on usage time and patterns to identify the technology components that affect (non)use [37].

The studies in our review suggest that ECAs can make an eHealth intervention, aimed at improving people’s lifestyle, more engaging. This is possibly because of the capacity of ECAs to establish and maintain an empathic relationship [3]. However, one can wonder how lasting this engagement is. Providing an ECA may have a novelty effect; thus, the engaging effect may wear off over time, resulting in decreased adherence, which is common for eHealth interventions [10]. Studying the use, effectiveness, and user experience of working with an ECA for coaching people in a healthy lifestyle for a prolonged period and in a realistic setting would provide inputs for answering these questions. Both researchers and eHealth developers need to find these answers to identify the persuasive goals that ECAs can serve best and to know how such ECAs should be developed to create engagement and a lasting effect.

On the basis of the findings of this review, we formulate several recommendations for future design and research. With respect to the development of ECAs for coaching people in a healthy lifestyle, we recommend the use of human-centered, stakeholder-inclusive design approaches, as well as reporting on the design activities in a systematic and comprehensive manner. This will allow others to learn from previous efforts. With respect to evaluation, there is a need to open the black box that is now pervasive among studies that delve into the efficacy of ECAs in improving health-related lifestyle. This means that evaluation reports need to specify which features are considered the main components of the eHealth intervention with an ECA and what theoretical foundation lies beneath these features, the ECA, and its persuasive tactic. Thereafter, during the data analysis phase of an evaluation, these features should be linked to measures of efficacy, use, and the user experience, to grasp whether the ECA works and why (not). Only in this way, a single evaluation can become valuable, both within and beyond its specific context.

Besides these general recommendations, we have also identified several specific research questions. As we mentioned in the introduction, the 24×7 availability of an ECA and its potential to deliver coaching at exactly the right moment (ie, just before or after specific behavior) make it a potentially valuable addition to the persuasive tool kit that eHealth developers have at hand. However, none of the included studies focused on identifying the exact right timing for a specific type of content. Should we always try to prevent negative behavior, thereby running the risk that the ECA may become annoying? Should we always acknowledge positive behavior, thereby running the risk that the ECA loses credibility? Finding the answers to these questions related to timing and frequency of use will allow us to create persuasive tactics for ECAs, which are in line with the tolerance levels and needs of end users. Furthermore, to fully understand the novelty effect that the introduction of an ECA may bring and to grasp the development of behavior change over time, longitudinal studies need to be performed. Ideally, these studies are (partly) in depth and qualitative to generate hypotheses for a novel field that can then be confirmed in large-scale quantitative studies afterward.

The first limitation is that we might have missed relevant studies. The applied search strategy might have influenced our findings, as it is plausible that ongoing studies are only published in conference proceedings. The applied search string might also have influenced our findings. During the stage of identifying relevant keywords, we already found a variety of terms used to describe (comparable) ECAs. With the help of a librarian, we therefore tried to mitigate this risk by setting up a comprehensive list based on an initial search. In the end, we identified 9 different terms in the studies included, although the definitions were rather similar. As a recommendation for future work, we propose to use the term ECAs as the uniform term for “more or less autonomous and intelligent software entities with an embodiment used to communicate with the user” [2].

The second limitation relates to the identification of BCTs. They were rather difficult to identify as they were often mentioned summarily in the text or within images, and no uniform language was used, for example, we could only code Tailored strategies that addressed related barriers [31] as problem solving, according to the BCT Taxonomy (v1) from the study by Michie et al [16]. Further descriptions were usually not provided.

ECAs are a promising tool for persuasive communication in the health domain. This scoping review provided valuable insight into the current development processes and evaluation outcomes. On the basis of these results, we offer multiple recommendations for future research agendas. We hope that the lessons from this review will further shape the novel field of using ECAs within the eHealth context.