Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 30.08.19 in Vol 21, No 8 (2019): August

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/13769, first published Feb 21, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Perspectives From Authors and Editors in the Biomedical Disciplines on Predatory Journals: Survey Study

    Department of Urology, University of California-San Francisco, San Francisco, CA, United States

    Corresponding Author:

    Benjamin N Breyer, MD

    Department of Urology

    University of California-San Francisco

    400 Parnassus Avenue, Suite A610

    San Francisco, CA, 94110

    United States

    Phone: 1 415 476 3372

    Email: Benjamin.Breyer@ucsf.edu


    ABSTRACT

    Background: Predatory journals fail to fulfill the tenets of biomedical publication: peer review, circulation, and access in perpetuity. Despite increasing attention in the lay and scientific press, no studies have directly assessed the perceptions of the authors or editors involved.

    Objective: Our objective was to understand the motivation of authors in sending their work to potentially predatory journals. Moreover, we aimed to understand the perspective of journal editors at journals cited as potentially predatory.

    Methods: Potential online predatory journals were randomly selected among 350 publishers and their 2204 biomedical journals. Author and editor email information was valid for 2227 total potential participants. A survey for authors and editors was created in an iterative fashion and distributed. Surveys assessed attitudes and knowledge about predatory publishing. Narrative comments were invited.

    Results: A total of 249 complete survey responses were analyzed. A total of 40% of editors (17/43) surveyed were not aware that they were listed as an editor for the particular journal in question. A total of 21.8% of authors (45/206) confirmed a lack of peer review. Whereas 77% (33/43) of all surveyed editors were at least somewhat familiar with predatory journals, only 33.0% of authors (68/206) were somewhat familiar with them (P<.001). Only 26.2% of authors (54/206) were aware of Beall’s list of predatory journals versus 49% (21/43) of editors (P<.001). A total of 30.1% of authors (62/206) believed their publication was published in a predatory journal. After defining predatory publishing, 87.9% of authors (181/206) surveyed would not publish in the same journal in the future.

    Conclusions: Authors publishing in suspected predatory journals are alarmingly uninformed in terms of predatory journal quality and practices. Editors’ increased familiarity with predatory publishing did little to prevent their unwitting listing as editors. Some suspected predatory journals did provide services akin to open access publication. Education, research mentorship, and a realignment of research incentives may decrease the impact of predatory publishing.

    J Med Internet Res 2019;21(8):e13769

    doi:10.2196/13769

    KEYWORDS



    Introduction

    Increased access to the Internet has allowed for open access publishing to flourish. Traditional modes of scholarly publication involve the transfer of copyright from authors to publishers, with journal fees collected to provide access to articles. Traditional publishing, whether due to cost or perceived prejudicial peer review, is not embraced by all [1,2]. In contrast, in an open access model, authors typically retain rights to their work, it is immediately available to all readers, and funding is provided by authors themselves in the form of publication fees. Almost exclusively, publication and circulation of open access articles occurs through the Internet. Both models involve peer review, editing, and some degree of article promotion.

    So called “predatory” journals take advantage of the open access publication model to prey on unsuspecting authors [3]. Such journals advertise the same services as open access journals but may fail to provide adequate peer review, licensing, quality control, and content preservation. Predatory journals are more likely to solicit authors via email for papers in exchange for publishing fees. Given that monetary incentives comes from author submissions, they are not beholden to the same motivation of high, perceived journal quality to entice library subscription as in traditional journals. By design, early-career physicians may not suspect such journals as illegitimate and are strongly incentivized to publish to advance their careers [4]. Predatory journals’ articles are not typically indexed on accepted forums and are objectively poor venues for dissemination. Predatory journals go to great lengths to appear legitimate; the fact that 400,000 items are published online per year under their banner speaks to their success [5].

    Defining what is and is not a predatory journal has been a point of contention among researchers [6-8]. Beall’s list, a list of publishers thought to be possibly predatory based on a single researcher’s criteria, has been controversial [8]. It is now only available as an online resource. Recently, other groups have independently identified characteristics that suggest a journal is predatory [9]. Lists that purposefully provide certification of legitimate open access publishing, such as the Directory of Open Access Journals (DOAJ) [10] and the Open Access Scholarly Publishers Association (OASPA) [11], offer an alternative way to ensure high-quality open access publishing. Up-and-coming open access journals could appear to share some of the characteristics of a predatory journal and have been critical of blacklists [7,12].

    What remains unknown amid this controversy is the perspective of both editors who manage predatory journals and those choosing to publish in them. While the lay press and literature has increasingly drawn attention to this issue, these journals continue to exist [3,5,13]. No literature to date addresses whether authors are unknowing victims or complacent coconspirators in the predatory publishing scheme. Our objective was to understand the motivation of authors in sending their work to potentially predatory journals. Moreover, we aimed to understand the perspective of journal editors at journals publicly cited as potentially predatory. We hypothesize that authors and editors are unaware that journals in which they are involved are possibly predatory.


    Methods

    Beall’s list [8] was accessed on August 1, 2018, at which time 2567 publishers were noted. Beall’s list was created using 48 criteria for a publisher-at-large or an individual journal [8]. The criteria assess varied factors such as grammatical or spelling errors on official communications, availability of complete editorial contact information, and the presence of false claims of indexing by services such as PubMed [14]. At present, this list is exclusively available online [8].

    A total of 350 publishers were randomly selected from this list, using a random-number generator. Eight reviewers (AC, GP, PK, MN, AT, JM, CB, and SW) evaluated publishers using the following criteria. First, we confirmed an active link to the publisher website. Second, we reviewed the titles of journals within the portfolio of a publisher’s work to assess if any were biomedical journals in scope. Specifically, we applied the MEDLINE criteria: “[Journals] predominantly devoted to reporting original investigations in the biomedical and health sciences, including research in the basic sciences; clinical trials of therapeutic agents; effectiveness of diagnostic or therapeutic techniques; or studies relating to the behavioral, epidemiological, or educational aspects of medicine” [15]. Journals meeting this criteria were noted. Third, we cross-referenced listed journals with the DOAJ [10], the OASPA [11], and the US National Library of Medicine [14]. We excluded journals listed on any of these sites. Fourth, we assessed whether links to specific journals were live and whether the journals had published articles. Fifth, the reviewers selected the most recent, original online article from the journal and recorded the name, country of origin, and email of both author and editor from the website. The selection criteria and ultimate composition of the cohort are summarized in Figure 1. All data were stored in REDCap [16].

    Surveys for both authors and editors were created during a round-table discussion among study authors. The surveys underwent iterative testing for clarity, content, and length. Next, we invited editors and authors to participate via email. Surveys were administered using REDCap, with one reminder sent to nonresponders. We simultaneously performed a nested randomized controlled trial on incentives to encourage survey response, which is published separately [17]. Invitees were randomized into a control group, a group eligible for a cash prize (US $100), and a group whose response lead to monetary donation to charity (US $2.50 to Rotary International per respondent). Rotary International is a nondenominational international charity with 35,000 worldwide clubs that have been instrumental in multiple projects, including the fight to eradicate Polio [18]. To maximize recruitment, survey invitations were personalized to include the article and journal name for authors and the journal name for editors.

    Surveys assessed authors’ basic demographics, publication history, their recollection of the editorial process for the article in question, and their knowledge regarding predatory journals (see Multimedia Appendix 1). Editors were queried about basic demographics, the editorial process for the publication in the journal in question, cost of publication, and their knowledge regarding predatory journals (see Multimedia Appendix 2). Of note, the mention of predatory journals was not made until a separate page of the survey, so as to not prejudice responses. For the purposes of this study, we defined a predatory journal as “an exploitative open-access academic publishing business model that involves charging publication fees to authors without providing the editorial and publishing services typically associated with legitimate journals.” Citations of articles as reported by authors were confirmed using a search in Google Scholar [19]. Partial survey responses were excluded, but those answering Prefer not to answer were not considered incomplete responses.

    Figure 1. The CONsolidated Standards Of Reporting Trials (CONSORT) flow diagram. DOAJ: Directory of Open Access Journals; OASPA: Open Access Scholarly Publishers Association.
    View this figure

    Summary statistics were used to describe the cohort. Means and standard deviations as well medians and interquartile ranges (IQRs) were used for continuous variables. Frequency tables were used for categorical variables. The chi-square statistic was used to compare frequencies between groups. Developed-nation status was based on the World Bank listing for high-income countries derived from gross national income higher than US $12,056 per capita [20]. Statistics were calculated using Stata 15 (StataCorp). The datasets used and/or analyzed during this study are available from the corresponding author upon reasonable request.

    Consent for publication was granted by participants when they responded to the survey in accordance with Institutional Review Board (IRB) approval. Ethical clearance was granted by the IRB of the University of California, San Francisco, CA (approval number: 18-25351).


    Results

    Overview

    Journals meeting selection criteria came from 181 distinct publishers. There was a substantial range in the number of journals per publisher represented in our cohort. A total of 58.0% of publishers (105/181) were singular entities, having only a single journal in their portfolio. However, several larger publishers were also included; indeed, 52.00% of (1146/2204) journals came from just 3 out of 181 publishing companies (1.7%). Of the articles selected, 59.68% (811/1359) were published in 2018, 17.73% (241/1359) were published in 2017, and none were published before 2014. Authors and editors represented a global academic community (see Figure 2). Of all articles selected, 40.03% (544/1359) had corresponding authors from high-income countries, as defined by the World Bank [20]. The overall survey response rate was 13.0%, with editor response rates significantly lower than those for authors (6.5% vs 18.9%, respectively, P<.01). Several respondents provided incomplete responses (n=40). Complete responses were collated from 206 authors and 43 editors, out of a potential 1165 and 1062, respectively, for a final response rate of 11.18% (249/2227).

    Authors

    Responding authors had a median age of 43 years (IQR 33-54), with 47.0% (95/202) reporting that they had been in practice for over 15 years (see Table 1). A total of 7.4% (15/202) of authors self-identified as still in a training program. Among authors, 80.1% (165/206) reported that publication of articles is part of their academic promotion process. Authors published a median of 3 (IQR 2-6) manuscripts in the year prior to survey completion; lifetime median articles published was 15 (IQR 5-45). A total of 56.1% (110/196) of funding for research projects was from personal funds. Authors reported a median cost of US $190 (IQR 0-520) for publication. A total of 78.2% of authors (158/202) recalled a peer review, with 38.6% (76/197) reporting 16-30 days between submission and acceptance. A total of 68.0% of authors (140/206) had to submit revisions and 67.0% of authors (138/206) did not submit their article elsewhere before selecting this particular journal. Of 206 studies, 22.3% (n=46) were observational, 11.2% (n=23) were basic science studies, 10.7% (n=22) were case series, and 9.7% (n=20) were systematic reviews.

    A total of 12.1% of authors (25/206) felt publication in their chosen journal was both prestigious and had a positive impact on their career. In addition, 19.6% of authors (39/199) noted a positive career impact if they reported their paper was cited versus 10.6% (21/199) that reported a positive impact when their paper was not cited (P=.04), with 27.2% (56/206) of total articles reportedly cited. Authors’ perceptions of the journals’ prestige was not impacted by whether their paper was cited. Google Scholar searching confirmed that only 16% (9/56) of those articles purported to be cited had a recorded citation. A total of 40.8% of authors (84/206) felt this particular publication was neutral in terms of career advancement and 14.1% (29/206) had published in the same journal prior to this particular publication. Most common reasons for selecting this particular journal for publication were open access (74/206, 35.9%), solicitation (52/206, 25.2%), and affordability (31/206, 15.0%).

    Authors that paid fees in the top quartile (>US $519) for publication had no difference in perception of journal prestige, impact on career, shorter publication times, or fewer revision requests than authors paying lower fees (see Table 2). They were significantly more likely to use private, nongovernment funds (P<.01). These authors reporting higher expenses were not necessarily more likely from high-income countries (80/206, 38.8% vs 60/206, 29.1%; P=.08). Authors came from 54 countries, spanning all continents except Antarctica; 23.3% (48/206) were from India and 13.6% (28/206) were from the United States.

    Editors

    Editors’ median age was 41 years (IQR 33-53) (see Table 1). Among survey responders, 40% of editors (16/40) were from India, 20% (8/40) were from the United States, and the residual editors were from varied locations. Editors reported performing editorial duties for a median of 2 journals (IQR 2-4), and for 79% of the editors (34/43), such involvement promotes their academic advancement. A total of 42% of editors (18/43) work primarily in an academic center and 79% (34/43) noted that being an editor positively impacts their career.

    Figure 2. Geographic distribution of survey invitees and responses.
    View this figure
    Table 1. Authors’ and editors’ basic demographics.
    View this table
    Table 2. Publication costs and authors’ perceptions.
    View this table

    Editors’ and Authors’ Views on Predatory Journals

    A total of 40% of editors (17/43) who responded were not aware that they were listed as an editor for the particular journal in question. These editors were excluded from subsequent analysis. Among remaining editors, 96% (25/26) felt their journal was not a predatory journal and 50% (13/26) were already aware their journal was listed on Beall’s list. A total of 92% (24/26) of editors reported that their journal requires peer review. Editors reported that the mean cost to publish in their journals was US $634 (IQR 75-1360). Editors’ estimated, on average, that 50% of manuscripts required revisions and that 35% were rejected. A total of 54% (14/26) of editors reported that 31-45 days elapsed between submission and acceptance.

    Whereas 77% of all surveyed editors (33/43) were at least somewhat familiar with predatory journals, only 33.0% (68/206) of authors were somewhat familiar with them (P<.001) (see Figure 3). Similarly, only 26.2% of authors (54/206) were aware of Beall’s list of predatory journals versus 49% (21/43) of editors (P<.001). A total of 30.1% of authors (62/206) believed that their publication was published in a journal that could be defined as predatory, once given the definition. Given knowledge that a journal was definitely predatory, 87.9% of authors (181/206) surveyed would not publish in the same journal in the future.

    Figure 3. Discrepancy in author and editor opinions on predatory journals.
    View this figure

    Narrative comments were also collected via the survey. These provided surprising and conflicting insight into the attitudes and opinions of both authors and editors. Additionally, many survey respondents felt passionate enough about this topic to contact the research staff via email to share additional long-form opinions. Selected comments are presented in an anonymized fashion in Multimedia Appendix 3. Some editors expressed surprise and outrage about their listing as editors, whereas others were defensive of their journals, described continued editorial improvements, and in rare cases disparaged Beall’s list. Authors expressed regret about their publication choice, whereas others felt they had no other options or provided negative commentary about traditional publishing models. The comments emphasize the controversial and divergent opinions surrounding predatory publishing. These issues will not be solved overnight; as stated by one author, “In [country name], one must have two or three publications in [a] journal with impact, so you do not say what or why; simply you must do it.”


    Discussion

    Principal Findings

    We found editors were at least somewhat familiar with predatory journals, but the vast majority of authors (67%) were not. Per authors and editors, in limited cases, predatory journals seemingly provide some editorial support via revisions, rejections, and circulation that was enough to drive citations. Predatory journal authorship is a global phenomenon in our study, with higher penetrance in India and the United States. Alarmingly, 39% of editors reported not even being aware of being a journal editor for the journal in question. Several editors sent us comments stating they had previously asked that their name and contact information be removed from the journal websites, given no purposeful affiliation. After alerting authors that their recent publication was in a potentially predatory journal, 88% would avoid the same journal, demonstrating that via dissemination of knowledge regarding predatory practices, predatory publishing may decrease.

    Services Provided by Predatory Journals

    On the surface, these predatory journals are providing at least some service to authors. In our cohort, authors frequently recalled a peer review and a need to submit revisions. Editors similarly stated that a peer review was performed and approximately 35% of articles were rejected. A potential marker of journal prestige and circulation is ultimately citation, reported by 27% of surveyed authors [21]. Of note, we could only verify 16% of author-reported citations. While assessing the quality of dissemination or peer review is beyond the scope of this work, from provided comments, there is clearly a stigma with being associated with a predatory journal for both authors and editors. There is little debate that any journal continuing to list editors that renounce that affiliation are illegitimate. Nevertheless, this work urges caution in branding a journal predatory without insight provided by editors and authors; ongoing work defining predatory journals should incorporate their perspectives.

    Further Analysis

    We found that 40% of authors were from countries designated by the World Bank as high income; prior work suggests that authors who published in predatory journals were primarily from developing nations [22]. A predominance of Indian (34.7%), African (16.4%), and Asian (25.6%) authorship was shown previously, using a complex stratified selection criteria of publishers; in our cohort, 23% of authors were from India [5]. Recent growth in research enterprise in nations such as India has created a need for forums of publication. Publication in the mainstream scientific press may be limited by access, bias against international research work, lack of shared scientific interest, or a true marker of low-quality work [12]. India may be a case study for this situation and this could explain why it is overrepresented in authors and editors in predatory journals, as typical journals do not and cannot accept the quantity of work produced [23].

    Prior work proposed that young naïve authors are the target of predatory journals, but 47% of responding authors in this study have been in practice for more than 15 years [4]. Similarly, in economics, a surprising 11% of 1284 articles in predatory journals were published by authors registered as part of a prestigious international authorship group [24]. By way of strategy, publishers target emerging markets rather than new researchers specifically [25]. By creating multiple publishing sites, each with 60-100 unrelated journals tailored to a particular culture, it can be difficult to complete an online journal search and not link to one of these journals. Further analysis suggested that such journals essentially had the same editorial team, despite glaringly different topic areas and very few, if any, publications [25].

    A high proportion of responding authors in our cohort reported that publication is an important facet of academic advancement. There is a growing concern that the motivation to publish may supersede desires for research quality [26]. Both authors and editors are incentivized to publish and edit to advance their careers. While the scholarly advancement of knowledge is a noble pursuit in and of itself, a growing number of scientific articles are never cited, suggesting inherently that they are flawed, do not contribute to knowledge, or are in poorly indexed journals rendering their research essentially invisible [27]. It is a fact that due to limited resources and human capital, not all academic centers are set up to succeed as a research enterprise. This may limit the potential impact of research from such sources. Mixed with incentives to “publish or perish,” authors may be tempted to lower their standards for publication and provide a market for continued predatory publishing.

    Potential solutions to predatory publishing include demanding promotion committees judge faculty on the quality of publications in lieu of sheer number [27]. Authors should receive training regarding the existence of predatory publishers. The onus remains on authors to independently verify credentials of journals and to confirm journal indexing claims. New initiatives such as Think. Check. Submit. [28], an international collaboration providing practical resources to educate researchers, promote integrity, and build credible research publications, should be publicized [29]. Research mentors should assist colleagues and steer them toward legitimate journals. White lists or other affirming criteria should be publicized among authors seeking publication to motivate higher-quality outlets for their work. Authors in our study were most motivated by cost and the open access model, so high-quality journals meeting that criteria should be lauded. They were also motivated by solicitations, so education surrounding such email or print invitations should be circulated [30]. Accountability for publishers should be demanded, particularly for blatant false claims involving indexing, editors, and cost. Regulatory oversight in the country of origin of predatory publishers should be strengthened—in some cases, no such oversight exists—and criminal charges pursued, if relevant.

    Some authors shared already-published concerns regarding blacklists, such as Beall’s stifling of innovation in open access publishing, which is why we cross-referenced our findings with well-publicized white lists to generate a cohort of authors and editors [6-8]. Prior work on predatory journals has not made this effort [5-8]. White lists provide an independently verified listing of journals with ethical and quality publication standards. We elected to use the US National Library of Medicine [14] as one white-list source in lieu of Embase or other publisher-operated sites. Given that Embase and similar sites are publishing-company owned, their ability to assess the suitability of journals for inclusion may be biased, given their own financial incentives. In contrast, the US National Library of Medicine has transparent requirements for inclusion. Nonetheless, different white lists could provide a slightly different cohort of authors or editors to study and, hence, caution is advised in drawing generalizations.

    Limitations

    A major limitation to this study is our poor response rates from both authors and editors, but particularly editors. Given that 39% of responding editors were not even aware that they were editors at the journal in question, it is not surprising that editors-at-large may have faced confusion with our survey invitation. Moreover, unlike author email addresses, the editorial email addresses often took a generic form, such as “editor@journalname.org,” and may have encountered staff that did not forward the survey to the intended recipient. Our data comes from an international cohort, but we randomized the selection of publishers and not the countries of publishers. As such, some countries only contributed via a single survey response and the responses should not be seen as representative of all authors of that region. All authors published their papers in English journals. Nonetheless, English may not be a first language for respondents, limiting responsiveness or leading to response error. Due to monetary and time constraints, we did not perform survey adaptation or cross-cultural validation for global participants [31,32]. Survey response may have been low due to survey fatigue, disinterest, or prejudice against the topic area, which may have increased response bias [31,32]. Ultimately, this was a volunteer sample, which selected for respondents over nonrespondents. Given that our demographic data exclusively came from voluntary respondents, we cannot compare to the nonresponder group, reducing the generalizability of our work.

    Conclusions

    Predatory journal authorship is a global phenomenon not unique to early-career researchers. The majority of studied authors were not familiar with predatory publishing practices, despite being published in a suspected predatory journal. Alarmingly, 39% of editors reported that they were not even aware of being an editor for the journal in question, clearly confirming the unethical practices of such journals. Education, research mentorship, and a realignment of research incentives may decrease the impact of predatory publishing.

    Acknowledgments

    Kelly Johnson provided invaluable assistance in supporting this research. Anthony Enriquez is recognized for his tireless efforts in assisting with IRB approval. We acknowledge the generous philanthropic support of the Alafi Fund, which was awarded to BNB and supported this work.

    Authors' Contributions

    BNB and AC conceived and conceptualized the study, coordinated the data collection activity, carried out the statistical analysis, and wrote the initial draft of the manuscript. SW, CB, PK, GP, AT, JM, MN, and AC collected the raw data, including contact information for potential survey respondents. They also iteratively developed the survey tool and participated in the design of the study, as well as edited the manuscript. PK specifically assisted with methodology development. GP was instrumental in editing the initial drafts. GP also assisted with graphic design and overall scope of the work. AC and GP coordinated the REDCap database housing the data and created the survey invitations. BNB provided mentorship and leadership for the project. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Author online survey.

    PDF File (Adobe PDF File), 43KB

    Multimedia Appendix 2

    Editor online survey.

    PDF File (Adobe PDF File), 44KB

    Multimedia Appendix 3

    Free-form comments, edited for length, grammar, and spelling.

    PDF File (Adobe PDF File), 297KB

    References

    1. Teixeira da Silva JA, Dobránszki J. Problems with traditional science publishing and finding a wider niche for post-publication peer review. Account Res 2015;22(1):22-40. [CrossRef] [Medline]
    2. Eriksson S, Helgesson G. The false academy: Predatory publishing in science and bioethics. Med Health Care Philos 2017 Jun;20(2):163-170 [FREE Full text] [CrossRef] [Medline]
    3. Moher D, Shamseer L, Cobey K, Lalu M, Galipeau J, Avey M, et al. Stop this waste of people, animals and money. Nature 2017 Sep 06;549(7670):23-25. [CrossRef] [Medline]
    4. Cariappa M, Dalal S, Chatterjee K. To publish and perish: A Faustian bargain or a Hobson's choice. Med J Armed Forces India 2016 Apr;72(2):168-171 [FREE Full text] [CrossRef] [Medline]
    5. Shen C, Björk BC. 'Predatory' open access: A longitudinal study of article volumes and market characteristics. BMC Med 2015 Oct 01;13:230 [FREE Full text] [CrossRef] [Medline]
    6. Strielkowski W. Predatory journals: Beall's List is missed. Nature 2017 Apr 26;544(7651):416. [CrossRef] [Medline]
    7. Teixeira da Silva JA. Caution with the continued use of Jeffrey Beall's "predatory" open access publishing lists. AME Med J 2017 Jul;2:97. [CrossRef]
    8. Beall's List of Predatory Journals and Publishers.   URL: https://beallslist.weebly.com/
    9. Shamseer L, Moher D, Maduekwe O, Turner L, Barbour V, Burch R, et al. Potential predatory and legitimate biomedical journals: Can you tell the difference? A cross-sectional comparison. BMC Med 2017 Mar 16;15(1):28 [FREE Full text] [CrossRef] [Medline]
    10. DOAJ (Directory of Open Access Journals).   URL: https://doaj.org/
    11. Open Access Scholarly Publishers Association (OASPA).   URL: https://oaspa.org/
    12. Stojanovski J, Marušić A. Does small equal predatory? Analysis of publication charges and transparency of editorial policies in Croatian open access journals. Biochem Med (Zagreb) 2017 Jun 15;27(2):292-299 [FREE Full text] [CrossRef] [Medline]
    13. Neuroskeptic. Discover Magazine. 2017 Jul 22. Predatory journals hit by 'Star Wars' sting   URL: http:/​/blogs.​discovermagazine.com/​neuroskeptic/​2017/​07/​22/​predatory-journals-star-wars-sting/​#.​WsQntejwZnJ
    14. National Center for Biotechnology Information (NCBI). PubMed   URL: https://www.ncbi.nlm.nih.gov/pubmed/ [accessed 2019-08-01]
    15. US National Library of Medicine (NLM). Fact sheet: MEDLINE® journal selection   URL: https://www.nlm.nih.gov/pubs/factsheets/jsel.html [accessed 2018-08-01]
    16. Harris P, Taylor R, Thielke R, Payne J, Gonzalez N, Conde J. Research electronic data capture (REDCap): A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
    17. Cohen A, Washington S, Butler C, Kamal P, Patino G, Tresh A, et al. Altruistic donation to improve survey responses: A global randomized trial. BMC Res Notes 2019 Feb 28;12(1):113 [FREE Full text] [CrossRef] [Medline]
    18. Rotary International.   URL: https://www.rotary.org/en [accessed 2019-06-19]
    19. Google Scholar.   URL: https://scholar.google.com/ [accessed 2019-06-20]
    20. World Bank Data Help Desk. World Bank country and lending groups   URL: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519#High_income
    21. Leydesdorff L, Bornmann L, Comins JA, Milojevi S. Citations: Indicators of quality? The impact fallacy. Front Res Metr Anal 2016 Aug 02;1:1-15 [FREE Full text] [CrossRef]
    22. Xia J, Harmon J, Connolly K, Donnelly R, Anderson M, Howard H. Who publishes in predatory journals? J Assoc Inf Sci Technol 2015 Jul;66(7):1406-1417. [CrossRef]
    23. Satyanarayana K. Time for 'Publish in India' movement. Indian J Med Res 2004 May;119(5):vii-vix. [Medline]
    24. Wallace F, Perry T. Munich Personal RePEc Archive (MPRA). 2016 Jun 01. Economists behaving badly: Publications in predatory journals   URL: https://mpra.ub.uni-muenchen.de/73075/1/MPRA_paper_73075.pdf [accessed 2019-08-01]
    25. Bolshete P. Analysis of thirteen predatory publishers: A trap for eager-to-publish researchers. Curr Med Res Opin 2018 Jan;34(1):157-162. [CrossRef] [Medline]
    26. Rawat S, Meena S. Publish or perish: Where are we heading? J Res Med Sci 2014 Feb;19(2):87-89 [FREE Full text] [Medline]
    27. Dahlia R. London School of Economics (LSE) Impact Blog. 2014 Apr 23. Are 90% of academic papers really never cited? Reviewing the literature on academic citations   URL: http://blogs.lse.ac.uk/impactofsocialsciences/2014/04/23/academic-papers-citation-rates-remler/
    28. Think. Check. Submit.   URL: https://thinkchecksubmit.org/ [accessed 2019-06-25]
    29. Beall J. Best practices for scholarly authors in the age of predatory journals. Ann R Coll Surg Engl 2016 Feb;98(2):77-79 [FREE Full text] [CrossRef] [Medline]
    30. Clemons M, de Costa E Silva M, Joy A, Cobey K, Mazzarello S, Stober C, et al. Predatory invitations from journals: More than just a nuisance? Oncologist 2017 Feb;22(2):236-240 [FREE Full text] [CrossRef] [Medline]
    31. Epstein J, Santo R, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol 2015 Apr;68(4):435-441. [CrossRef] [Medline]
    32. Lee J, Jung D. Measurement issues across different cultures. Taehan Kanho Hakhoe Chi 2006 Dec;36(8):1295-1300. [Medline]


    Abbreviations

    CONSORT: CONsolidated Standards Of Reporting Trials
    DOAJ: Directory of Open Access Journals
    IQR: interquartile range
    IRB: Institutional Review Board
    OASPA: Open Access Scholarly Publishers Association


    Edited by G Eysenbach; submitted 21.02.19; peer-reviewed by O Beiki, S Nimbalkar; comments to author 11.04.19; revised version received 24.04.19; accepted 07.05.19; published 30.08.19

    ©Andrew J Cohen, German Patino, Puneet Kamal, Medina Ndoye, Anas Tresh, Jorge Mena, Christi Butler, Samuel Washington, Benjamin N Breyer. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 30.08.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.