We identified 9 broader categories of the research topics in the 44 included studies (see Table 2.1 in Multimedia Appendix 2 for the full list). The most common were home care and telemonitoring, and health promotion and education (both n=8), followed by electronic medical records and health information systems (n=7), and mental health services (n=5).

The studies were set in 21 different countries, Australia being the most common (n=5) followed by the United States (n=4) and Canada, Sweden, and the United Kingdom (all n=3). Some studies from nonwestern countries, like Tanzania or Colombia were included, but no country was represented more than once or twice. Within the different countries, studies took place in various contexts, the most prevalent of which were rural areas (n=6) and hospitals (n=5). All contexts and countries can be found in Tables 2.2 and 2.3 of Multimedia Appendix 2.

Among the 44 studies, 2 studies explicitly focused on 2 different target groups at the same time, whereas all other studies had 1 main target group. In most cases, the target groups were patients (n=11). Of these, the most common group was patients with cancer (n=3). There were 6 studies each focusing on clinicians as well as children and young adults, and 5 studies targeted older adults (see Table 2.4 in Multimedia Appendix 2 for the full list of target groups).

In many cases, several stakeholders were included in the study, up to 6 different types of stakeholders included in some cases. In summary, 20 different types of stakeholders were involved (see Table 2.5 in Multimedia Appendix 2 for the full list). Health care workers (n=18) and patients and their representatives (n=12) were involved the most, followed by governmental bodies (n=9) and general nonmedical staff members (n=8). When clustering these stakeholder types according to the framework defined by Schiller and colleagues [17], the largest group consisted of practitioners and professionals (n=48), followed by members of the public (n=38). Policy makers and government bodies (n=13), the research community (n=10), private businesses (n=6), and civil society organizations (n=3) were represented less often. The only group that was not represented at all included health and social service providers.

Not all of the 44 studies reported the duration of the project (n=7). Studies that did report the duration (n=33) lasted from a few months (n=5) to more than 10 years (n=2). The majority (n=13) of these studies reported a project duration between 2 and 3 years, and the average project duration was 2.7 years. Figure 3 shows the distribution of the 10 most frequently involved types of stakeholders for the different project durations in the 33 projects that reported the project duration. Stakeholder types are shown in the order of how many times they were involved in total; however, because some studies did not report project durations, the numbers in this graph differ from those described above. The 2 biggest stakeholder groups, health care workers and patients, were rarely, or in the case of patients even not at all, involved in long-term studies.

In Figure 4, the study duration is mapped against the number of different stakeholders that were involved in each of the 33 projects that reported a project duration. Studies that did not report the overall project duration are not included in the figure. Most of the included studies lasted for up to 2 years, including 2 or 3 stakeholder groups. There are some longer studies including more stakeholder groups.

As mentioned earlier, AR is a framework that does not advise the use of a single methodology, and studies can therefore include a variety of different research methods. Most of the 44 included studies indeed used several methods, with some studies employing up to 6 different methods. Interviews were used most frequently (n=24), followed by focus groups (n=22), workshops (n=14), and surveys (n=13). On average, studies used nearly 3 different methods (average 2.8). All methods can be found in Table 2.6 of Multimedia Appendix 2.

The identified best practices in the 44 studies were most often related to the use of a specific method (n=9), namely personas (n=2), world café, journey mapping, role play, scenarios, case studies, design cards, and mixing different types of data collection methods (all n=1). Other best practices were a continuous evaluation of the project and a reflection on the process by the research team (n=8). The importance of establishing active contact between researchers and stakeholders and raising the confidence and skills of stakeholders was emphasized by 7 studies. The improvement of stakeholder skills mainly referred to research and analytical skills, allowing stakeholders to set up their own studies or continue the work after the project was finished. There were several specific suggestions to improve the regular project team meetings, for example, to always use the same agenda or to share a common area (office space) to make contact easier. Some other best practices concern the reporting and presentation of outcomes (n=6). The complete list of best practice categories can be found in Table 3.

Apart from the best practices, the lessons learned from each study were identified. The most common lessons learned were increasing stakeholder knowledge and skills (n=8) and continuous evaluation of the project and reflection on the process (n=6). Both of these had been identified as best practices in other articles (more on this overlap below). Recommendations for the use of specific methods were also common (n=5). Lessons learned regarding reporting, adapting the project to fit the needs of stakeholders, fostering a welcoming environment, and the questionable replicability of the research were each mentioned 4 times. All lessons learned are shown in Table 4.

As stated earlier, some aspects were identified as best practices in some articles and as lessons learned in others. In total, we identified 7 such overlapping aspects. Overall, the most mentioned aspect was the importance of raising stakeholder skills and confidence (n=15, where best practices= 7 and lessons learned=8). Many articles reported the need for stakeholders to learn new skills, for example related to academic research, or the need to be convinced about their ability to perform these tasks. Almost all the studies that reported this as a best practice or lesson learned involved health care professionals as stakeholders. Other commonly mentioned points were recommendations for specific methods, even though the suggested methods differed (n=14, where best practices=9 and lessons learned=5) and there was continuous reframing and evaluation of the project (n=14, where best practices=8 and lessons learned=6). Continuous reframing often referred to the iterations of planning, action, and evaluation in AR projects. Studies that described this mostly did not include this cyclical nature of AR in their definition of it. In total, there were 10 recommendations regarding the reporting and presentation of results (best practices=6 and lessons learned=4), for example calling for open and accessible publishing of outcomes. The best practices and lessons learned included recommendations about meeting regularly (n=9, where best practices=7 and lessons learned=2), adapting to the needs of stakeholders (n=8, where best practices=3 and lessons learned=5), and investing in the relationship between partners (n=6, where best practices=3 and lessons learned=3).

When observing the publication timeline, most of the overlapping aspects appeared as a lesson learned in earlier publications, and then as a best practice in papers published at a later point in time. This was the case regarding stakeholder skills, appearing as a lesson learned in 1999 [33] and as a best practice in 2016 [25]; continuous reframing of the project was a lesson learned in 2003 [19] and best practice in 2009 [42]; further, having regular meetings was a lesson learned in 2006 [72] and a best practice in 2018 [27], and adapting the research to stakeholder needs was a lesson learned in 2007 [32] and a best practice in 2016 [77]. Such a clear timeline could not be seen for accessible reporting, appearing as a lesson learned in 2017 [78] and a best practice in 2007 [45], and the relationship between partners appearing as a lesson learned in 2017 [36] and as a best practice in 2008 [38].

Being involved in a project as coresearcher can potentially increase stakeholders’ confidence, besides teaching them new skills [79]. However, this does not happen automatically. Similar to our findings, the narrative review conducted by Harrison and colleagues [80] also identified educating the research team as the most important task when stakeholders are involved in health care research. Nevertheless, there is limited research on how skill training for stakeholders could look like, and this can vary greatly between studies. Stakeholders in some eHealth studies might need to learn content-related information [81], whereas other studies require methodological or statistical skills [54]. Researchers should provide adequate training and material for their project and encourage stakeholders to make use of it. The studies included in this review that recommended stakeholder skill training almost exclusively worked with health care professionals. The relationship between recommending skill training and working mainly with health care professionals remains unclear. A possible explanation could be that other stakeholder groups in other studies already had the necessary skills and thus did not require any additional training. Another possibility is that other stakeholders were not given the same roles that health care professionals held, and therefore, they did not need skill training. Finally, as we will discuss later, reporting of AR activities was not always very extensive. Thus, stakeholders outside the health care sector were possibly trained, and these studies did not report on this aspect. Generally, not all participants prefer the same level of engagement in a project, and researchers should respect these preferences [82].

Different aspects of the role and tasks of the researcher in an AR project are discussed. Brydon-Miller and Aragón describe the many different tasks that action researchers need to fulfil as their “500 hats” [83]. These are not specific to eHealth studies, but they can occur in any AR study. As researchers and stakeholders have many varied duties, their roles are not fixed and might change over the course of the project [19]. One main task of the researchers that continues throughout the project is the need to foster a welcoming environment for all stakeholders [42]. Researchers should also be present and actively involve themselves at a higher level than that needed in non-AR projects [38]. Additional AR-specific tasks for the researchers include investing in partner relationships [35] or breaking down power structures [28]. Generally, AR studies demand more self-reflection and awareness from the researchers than other projects and researchers should keep this in mind when entering an AR project.

The importance of continuous reframing and evaluation of the project was emphasized in several studies. Although evaluation is 1 of the AR cycles, studies providing recommendations on this topic rarely included this in their definition of AR. Owing to the lack of reports on AR cycles, which will be discussed below, it is unclear if these studies still followed the AR cycles without reporting on them. However, sometimes, it seems that periodic planned evaluation is not enough. Instead, the participants need to regularly reflect on the current status of the project and their role in it. Therefore, new AR projects should create suitable spaces for evaluation and reflection in ways that fit the projects and stakeholders. This is especially important because reflection can become difficult once a person is in the middle of the project [49]. Holeman and Kane [53] emphasize that reflection should not only take place within the project, but it should also be explicitly reported to help other researchers. If action researchers take reflection seriously and include honest evaluations in their publishing, the AR community members can learn from each other. Additionally, researchers and other stakeholders within the project learn and benefit from constant reflection [9].

Another important aspect concerns paying attention to open and understandable dissemination of results within the community and among researchers. Action researchers need to communicate findings to the academic world while also finding ways to inform the target group about the project in ways that suit the target users’ needs. An example of open and accessible dissemination can be found in Canto-Farachala and Larrea [83]. They present the results of their AR project regarding territorial development on an interactive website, allowing others to learn from their work. However, it seems that accessible reporting is still not the norm in AR, as Avison and colleagues [62] describe that many AR studies are generally “published in books rather than as articles. Action researchers have large and complicated stories to tell.” Future AR projects should attempt to narrate their stories in such a way that others can learn from them.

The different way of describing AR studies also leads to another issue, incomplete and elusive reporting. Although most studies did provide at least a short description of what they saw as AR, 7 studies provided no definition at all. Additionally, there were only 4 studies that included 3 or all of the 4 aspects of the AR definition in their description. Even the most mentioned aspects appeared in less than half of the included papers. Even though most papers did cite an AR approach of definition, some did not. In combination with the often-limited descriptions of AR, this makes it difficult to obtain a clear picture of how AR is perceived and performed in a particular study. This resonates with what Bradbury and colleagues [9] describe as 1 of the quality points of AR, namely “action research process and related methods (should be) clearly articulated and illustrated.” The best practices and lessons learned that were extracted from the included studies were seldom mentioned explicitly. Best practices were often hidden in the description of the project, without much reasoning. Similarly, lessons learned were often described as adaptations made during the project or as plans for the future. Although we observed that some lessons learned turned into best practices over time, we think that researchers could benefit more from each other’s work by providing concrete recommendations. This review is a step in that direction. Both aspects show that the reporting of AR studies in eHealth can be improved to show more clearly what eHealth AR projects can look like and help others in setting up such projects with specific recommendations.