A detailed description of our peer-reviewed research protocol and conceptual framework can be found elsewhere [86]. This review was planned, conducted, and reported in adherence to standards of quality for scoping reviews [87,88]. A summary of our six-stage methodology follows.

Our research question was developed by consulting a group of knowledge users to determine their needs and questions about using collaborative writing applications for knowledge translation. We defined “collaborative writing applications” as a category of social media that enables the joint and simultaneous editing of a webpage or an online document by many end users (eg, wikis, Wikipedia, Google Knol, Google Docs, Google Sites) [21]. The participants targeted by this scoping review were health care stakeholders.

Seven scientific databases (Cochrane Library, PubMed, EMBASE, CINAHL, PsycINFO, ERIC, ProQuest Dissertations and Theses) were searched systematically for the period covering January 1, 2001 (Wikipedia’s inaugural year), to September 16, 2011. Our search strategy was peer-reviewed using the PRESS criteria [89]. The following keywords were used and adapted to each database: “wiki”, “wikis”, “Web 2.0”, “social media”, “Google Knol”, “Google Docs”, and “collaborative writing applications” (see Table 1).

We did not exclude any citations based on language. In addition, study reference lists; the 2010 and 2011 editions of the Medicine 2.0, WikiSym, and American Medical Informatics Association conference proceedings; clinicaltrials.gov and Open Medicine’s websites; expert consultation (eg, the authors of WikiProject Medicine [4]), OpenSIGLE (before 2005), and the Health Technology Assessment international Vortal were searched. Furthermore, environmental scans of the grey literature indexed by Google, Bing, Yahoo, and Mednar were performed. Finally, via email, Twitter, Mendeley, Google Docs, and a health librarianship page (HLWIKI), we called for the crowdsourcing of studies that could potentially fall within the scope of this review.

Three teams of 2 reviewers (SR/MF, TB/AB, PA/CK) independently screened titles, abstracts, and grey literature and retained articles that presented empirical data about any CWA applied to the field of health care. In case of disagreements, a third reviewer was consulted (PA, TB, or SR). To reach a high level of agreement, we conducted 4 series of assignments (400 abstracts in total) whereby the screening of a number of studies was followed by a teleconference to reach agreement about which studies to include and to discuss uncertainties. Once consensus was reached for all cases, the remaining studies were coded by the same 3 pairs of screeners (SR/MF, TB/AB, PA/CK). Subsequently, 2 reviewers (TB and PA) conducted another round of screening based on full text studies. As a result, a narrowed definition of health care was applied in order to focus the analysis. Hence, studies that concerned the care of patients were included, and those from the fields of basic medical sciences, the conduct of clinical trials, biomedical library science and medical informatics were excluded.

A data-charting form was developed and built into EPPI-Reviewer for the extraction of quantitative and qualitative variables and to facilitate data coding. It was tested and refined by 4 reviewers (PA, CN, ME, CF) using the first 50 studies. Three pairs of 2 reviewers (CN/CF, CN/ME, ME/CF) then independently extracted data from the remaining studies. Disagreements were resolved through discussion with a third reviewer (PA or TB). Using EPPI-Reviewer’s inductive coding function, we extracted all the pre-planned variables described in our published protocol [86].

As specified in our published protocol [86], we held meetings with representatives from the organizations involved (ie, the Association of Faculties of Medicine of Canada (AFMC), the International Medical Informatics Association (IMIA), the Federation of Patients and Consumer Organization in the Netherlands (NPCF), and the Pan American Health Organization (PAHO)) at the beginning, midway, and draft manuscript stages of this research in order to generate results that were useful for these knowledge users. Knowledge users were selected to represent a broad range of potential stakeholders representing medical education (AFMC), public health (IMIA and PAHO), and patient representatives (NPCF).

After removing duplicates (n=1372), we screened the title and abstract of 4436 citations as well as the studies/abstracts from the grey literature, conference proceedings, expert consultation, and reviewing of reference lists (Figure 1) . All disagreements (n=794) were resolved through discussion.

Crowdsourcing identified two studies through Google Docs that were excluded. After review, we included 111 citations. Among these 111 citations, there were 28 abstracts without published full text but with sufficient results to be included. Twenty-six studies were grouped into Theme 1 (use patterns of CWAs), 25 into Theme 2 (quality of information in different CWAs), and 73 into Theme 3 (use of CWAs as a knowledge translation intervention). Figure 2 shows the rapid growth of the number of publications for the period within our search strategy.

We confirmed that CWAs are currently being used frequently in health care, by a variety of stakeholders including patients, professionals, and researchers, for a large diversity of purposes. Our complete portrait of the literature shows that wikis are by far the most commonly studied type of CWA and that most studies had observational designs. Each type of CWA has different collaborative writing and conversational features that must be considered by decision makers when making a choice about which CWA to use in different collaborative projects. Many positive effects are attributed to the use of CWA in health professions education and knowledge translation. Further systematic synthesis of experimental and quasi-experimental evidence is needed before any clear policy recommendations can be made about implementing these tools in current practice. Moreover, there is an array of potential negative effects and barriers that need to be addressed in future primary research projects.

Despite the controversy surrounding the use of information in Wikipedia in clinical decision making [57,65], a high proportion of health professionals and students are already using Wikipedia and other CWAs, with use apparently increasing, especially among younger professionals. Although more research is needed to confirm this trend, these findings are consistent with an overall trend to increased use of social media among health professionals [79,272]. Our systematic mapping of the literature shows that wikis are the most frequently studied type of CWA. Furthermore, the use of Wikipedia by students and professionals represents the focus of many of our included studies. Google Docs studies come second, and we found only one study about Google Knol. This is not surprising since Wikipedia is the sixth most visited website worldwide and appears in top 10 results of search engines concerning health questions [125]. However, as readership of Wikipedia is rapidly changing, it is important to acknowledge that usage percentages depend not only on how you ask the question but also when you ask the question. Moreover, Google terminated the Knol project in 2011 despite interesting health projects using this platform including the PLOS Currents: Influenza project [273,274]. Besides the single publication we found about Google Knol comparing Knol to Wikipedia [96], there are no published accounts of Google’s reasons for closing and transferring Knol to the Annotum platform.

Based on the Medicine 2.0 map [2], we demonstrated that current CWAs in use are mainly oriented towards health students and professionals’ peer-to-peer interactions. In fact, use of CWAs is a major area of research in health education [275,276]. In particular, of the 4 experimental studies identified, 3 were education studies showing that CWAs positively influenced learning processes and almost half (n=48) of all the studies in this review concerned health professions education. Albeit less common, there are also studies about CWAs involving consumers and professionals to co-create decision-making tools [100,101,105,277]. These four projects seem relevant given that patient-centered care has become a central aspect of knowledge translation and experts have called for new ways of involving patients in the implementation of evidence [278]. Another remarkable finding is that even fewer CWAs involve consumers and researchers in sharing hard to find phenotype information about rare genetic and congenital diseases [106,144].

Researchers are starting to explore the use of CWAs, for example in updating a scoping review [130]. Another expert/researcher driven wiki is the OpenMRS electronic medical record implementation wiki, an example of wikis’ full potential for improving health in developing countries. Although the World Health Organization is exploring the use of a wiki to update the 11th International Classification of Disease [49], we did not find any published accounts on their experience, nor did we find any related to the discontinuation of Medpedia [37]. The reasons for ending this ambitious project involving important stakeholders would provide lessons for the future.

After comparing how each CWA was used in different collaborative writing projects, we found that wikis and certain hybrid custom-built wikis have collaborative writing features that are more prominent compared to their conversational features. These collaborative writing features produce artefacts of synthesized knowledge that lend themselves more readily to daily use than those produced from conversational knowledge. For example, using a wiki to store and update care protocols readily applicable to the care of emergency department patients would be more useful in daily practice than reading the discussion page found in support of the wiki page itself. Conversely, Google Docs, certain knowledge management applications (eg, Google Sites, Microsoft Sharepoint) and other social media platforms (eg, MijnZorgNet, Atlassian Confluence, MinJournal) integrate additional features that favor conversation and deliberation between users. These additional conversational features produce discussions between users about the knowledge being shared and add to users’ understanding about the content found on the collaborative writing pages of these applications.

Most evidence stemmed from case reports and observational studies demonstrating perceived positive effects of CWA use in health care on behavior change, education, communication, collaboration, knowledge management and access to knowledge, and better quality and efficiency of health care. These findings support claims that CWAs and wikis facilitate that online professional communities create, share, and synthesize knowledge; increase access to health information; and offer opportunity for public participation and citizenship [84,276,279]. Although less frequently reported, we also found a series of perceived negative effects (ie, information overload, fast dissemination of poorly validated information, loss of autonomy, feeling of working in isolation, increased stress, perceived unequal distribution of tasks within teams, biased editing, editing wars, and vandalism/wikispam) that could mask some of the positive effects of CWAs. Innovative developments such as semantic wikis [8,97,98,276,280] and bots [11,281] may decrease some of these negative effects. For example, to reduce the impression of information overload, certain authors are exploring semantic wikis to better organize and structure information based on a logical ontology [97,98]. Semantic wikis could help organize the knowledge being shared [8,276,280], potentially improve its meaningful use [282,283] and eventually allow its integration into intelligent Web-based decision-support tools [280]. Other authors are exploring the use of bots to decrease the risk of vandalism, biased editing, and spam [11,281]. A bot is a computer program that runs automatically and continuously within wikis and can conduct simple tasks like correcting spelling and syntax. Wikipedia contains many different bots that help ensure its quality [281]. More complex bots exist like the one in WikiPathways that surveys the content and identifies potential inconsistencies, redundancies, and incomplete data [11].

The use of CWAs in health care faces barriers that limit their use that are similar to those experienced in other fields: unfamiliarity with ICT [284], time constraints and workload [275], lack of self-efficacy to use CWAs [275], access to CWAs [285], worries about the scientific quality of the information resources [276,281,286,287], readability of information [281], the presence of a closed wiki protected by a password [276,281] and legal concerns [276,286,287].

A recurrent finding about the information in Wikipedia was that it is in large part accurate, free, and easy to access. However, even though Wikipedia does not recommend including medication doses due to concerns about errors [288], it is often incomplete and can lack appropriate referencing of medical information [58,61], thereby possibly indirectly causing patient harm [135]. One observational study demonstrated that involving moderators and experts in the sharing and curation of information within CWAs improves the quality of information [99]. However, as previous authors have demonstrated, finding ways to get these experts to participate remains a challenge [4,130,182,276,289].

Maintaining high-quality information as well as high contribution levels is a heated debate with opposing views (ie, password-protected wiki vs open wiki) [53,105,109,155,163,169,173]. Authors from multiple fields have explored modalities to stimulate participation [276,281,284,285,290-296]. Many facilitators reported from fields other than health care include training [284,296], scientific quality of the information resources [281,286,287], ease of use [291], having access to integrated support tools [296], ease of content editing [297-299], access to CWA [285], self-efficacy [300,301], and the use of incentives [293,294,302-304]. Some propose a set of scholarly metrics that would reward contributions to collaborative projects [130]. The journal RNA Biology stimulates contributions to Wikipedia by scholars by requiring that manuscripts be summarized for a Wikipedia page before accepting to publish the article [305]. The WikiGenes project has recognized the importance of authorship [10,36]. Finally, similar to other fields [293,294,297,306], the presence of a community and the sense of community is a frequently reported facilitator that increases contributions by health care stakeholders. Experts suggest that studying CWAs involves looking at both the technology and its community of users [276,285,307]. Thus, understanding the success of a project using a CWA must also include exploring the fundamental elements of communities of practice [266]. Communities of practice can meet online (ie, virtual community) or face to face. Similar to systematic reviews on communities of practice [308,309], our scoping review identified the presence of a moderator and/or a champion as a key factor for a successful collaborative writing project. Related to the concept of community, the success of a collaborative writing project also includes having a critical mass of participants, shared values, openness, trust, and respect.

We believe that our findings are important for consumers, professionals, researchers, and health care organizations around the world that are already using CWAs and/or planning to use a CWA to improve health care. Although we have found some evidence from experimental studies to support the use of CWAs as a health profession’s educational intervention and a large body of observational evidence supporting the use of CWAs as a knowledge translation intervention, a formal systematic review should be conducted to further synthesize the evidence and conduct a formal risk of bias assessment before making practice recommendations. Furthermore, the implementation of CWAs is fraught with barriers and the potential for adverse effects, requiring primary research to assess their safety.

Unfortunately, the breadth and depth of the literature on the use of CWAs specific to public health is scarce. However, based on some ongoing and promising projects [49,76,99,139,146,164], it is clear that the uses of CWAs for public health are vast and far-reaching. Although more research is needed within this specific domain, CWAs improve information access, collaboration, and can improve health education—all tenets of public health. Patients and consumers often experience many barriers in the use of CWAs, with information quality being among the most reported. The readability of articles within Wikipedia is a key area that must be addressed, as it will improve health literacy and knowledge translation [310]. There are also promising projects that may shed light on the effectiveness of involving patients in the development of clinical of guidelines [311]. Evidence from experimental studies about engaging patients with CWAs is still rare and needs to be replicated in robust prospective trials before making recommendations.

This is the first study that has conducted a scoping review to examine the depth and breadth of evidence about the use of CWAs in health care. We rigorously followed scoping review methodology and conducted a systematic and broad search of CWA use in multiple scientific databases and grey literature sources. A scoping review was the ideal methodology to employ for a number of reasons. First, it is an explorative method used when the relevant literature is considered to be broad and diverse [312]. Moreover, the study of these applications is an emerging field that is being examined with diverse methods [28,32,61], with different theoretical frameworks [29] and in different contexts [46,313]. We used a high-quality collaborative Web-based software to manage our review, to import studies, to extract data and to create reports. Every step of our review has been extensively described. By including knowledge users and policy makers, we have produced a relevant synthesis of the evidence targeting their needs. Based on empirical results, this scoping review has also extended an existing taxonomy of adoption determinants to the study of a social media application. The original taxonomy had been developed using a rigorous mixed-methods systematic review methodology [78]. Although our new extended taxonomy is very comprehensive, we believe that this level of detail was important to maintain in order to help future researchers explore the impact of these barriers and facilitators. Moreover, we have also created a new taxonomy of effects based on elements from other sociocognitive and organizational frameworks of change. Our use of the Donebadian framework was very useful because of its generalizability and overarching broad scope. Other more specific frameworks (eg, Theoretical Domains Framework) fit well within this overarching framework. Research should validate our two new taxonomies for future development, assessment, and implementation of other social media applications.

Even though we did everything possible to minimize publication bias by systematically and extensively searching for any sources of the grey literature presenting negative results (eg, including a lay media newspaper article [177]), we believe publication bias is not excluded. For example, we have not found published reports explaining the failed attempts at maintaining Google Knol or Medpedia. Many other CWAs sites have also disappeared over the course of the years without any clear explanations. In 2009, David Rothman had listed 69 medical wikis, many of which are now inactive or simply do not exist anymore [39]. Such reports describing the reasons for CWA failure would help generate important lessons for the advance of the science of collaborative writing.

Second, our scoping review methodology [87,88] did not include formal quality assessment. However, we classified studies based on the strength of their design in order to help us identify areas for primary research and those that produced sufficiently robust evidence for making recommendations.

Third, our scoping review was limited to reviewing CWAs using a definition that excluded related applications like blogs, microblogs, discussion forums, and patient communities (eg, PatientsLikeMe). Even though these social media applications are collaborative as well and share some common features with CWAs, we believe that it is important to study them separately to better understand each application’s impact and interaction with other social media.

Finally, our search strategy is limited to studies published between January 1, 2001, and September 16, 2011, while several more recent studies about CWAs have been published [263,272,314-319].

This scoping review has identified a number of research gaps. There is a need to conduct systematic reviews to further synthesize the results of experimental and quasi-experimental studies in the field of health professions education and to further synthesize evidence about implementation strategies addressing the different barriers identified. Given that the majority of the literature presently exists in the form of case reports with self-reported measurements, it is essential that further prospective trials with objective outcomes be conducted. Future trials should identify implementation processes that can be influenced by CWAs and how to measure them (possibly using Web metrics [130,167,276]) as intermediate outcomes of a complex knowledge translation intervention. In this respect, in addition to other frameworks defining evaluation plans of dynamic collaborative applications [320], our taxonomies of CWA adoption determinants and effects will help plan such trials. This will help researchers understand the different mechanisms of action at play leading to improved patient-oriented outcomes (quality of life, morbidity, mortality). Although the feasibility of conducting a randomized clinical trial to study the effectiveness of CWAs seems daunting, other complex interventions have been studied using this methodology [321].

Before conducting such trials, researchers and decision makers must reflect on defining the purpose of using a CWA as a knowledge translation intervention. Researchers must also find ways to adapt CWAs to the particular needs of different stakeholder groups (consumers, professionals, and researchers). Important barriers such as the quality of information contained in different wikis must be better addressed. As previous authors have stated [183,320], measuring the quality of user-generated content and its change over time is a challenging task requiring research [322]. Finding ways of assuring the scientific integrity of evidence within CWAs and recognizing authorship are significant stumbling blocks that need to be addressed for health care [102,114,130,171,176,323]. Studying each specific behavior involved in using CWAs (ie, to use, to contribute, to edit, to delete) with the help of theoretical frameworks will also help inform future interventions.

In addition to other technical considerations [324,325], future studies should explore the impact of collaborative writing and conversational features on information sharing and investigate what kind of knowledge (explicit vs tacit [266]) is shared. This could help knowledge users choose an appropriate CWA. As future communication tools, the impact of using different types of media embedded within CWAs (audio and video recordings) should also be explored. Finally, an important consideration to explore in future studies would be to determine the impact of using a closed vs an open CWA on the quality of the information found within the CWA and on the type of barriers experienced by users.

The prevalence of CWA use is high in various fields of health care, and they are used for a variety of purposes. They present many potential positive and negative effects as knowledge translation tools. Although we found some experimental and quasi-experimental evidence in favor of using CWAs as educational and knowledge translation interventions, the vast majority of included studies were observational case reports about CWAs being used by health professionals and patients. More research is needed to determine which stakeholders benefit the most from using CWAs, to address the barriers to their use, to find ways to ensure the quality of their content, to foster contributions, and to make these tools effective knowledge translation tools for different stakeholders. Answers to these questions are needed before clear policy recommendations can be made about the safe use of CWAs in health care.