The objects of this study include 2 groups of journals, representing authoritative journals in the field of clinical medicine and OAMJs. The selection criteria for authoritative journals were as follows: (1) JIF ranks high in the field; (2) selected journals are not review journals; and (3) academic authority is widely recognized. The selection criteria for representative OAMJ journals were as follows: (1) OA peer-reviewed journals that charge publication processing fees; (2) an annual average of more than 2000 OA research papers published on clinical medicine topics; (3) a 3-year impact factor within Q1 or Q2.

Through a search of relevant journals in the field of clinical medicine, we discovered that the impact factors of the British Medical Journal (BMJ), the New England Journal of Medicine, The Lancet, and the Journal of the American Medical Association (JAMA) ranked among the highest in impact factors between 2017 and 2019, with widely recognized academic authority [20-25]. We also found that BMJ Open, PLoS ONE, and Scientific Reports published more than 2000 open-access research papers on clinical medicine topics annually, establishing a certain degree of academic influence.

Based on these criteria, this study selected BMJ, New England Journal of Medicine, The Lancet, and JAMA as the authoritative journals and BMJ Open, PLoS ONE, and Scientific Reports as the representative OAMJs.

Because there is no disruptive innovation in review articles, and because Bornmann et al [26] suggest that the disruption index should only be calculated for research papers with at least 10 references, this study exclusively focused on clinical medical research papers with more than 10 references from the aforementioned journals.

The data required for this study included journal information, literature information, and citation relationship information, obtained through Journal Citation Reports (JCR), Web of Science (WoS), InCites, and the OpenCitations Index of PubMed open PMID-to-PMID citations (POCI), respectively. POCI, a data set sourced from the National Institutes of Health, comprises over 700 million citations and 29 million bibliographic resources. In POCI, citations are regarded as first-class data entities with 7 attribute fields, including the citation time span. To evaluate disruptive innovation, we selected focus papers and used their PMIDs along with POCI data identifiers to calculate the disruptive innovation level for each paper.

First, we logged into JCR, WoS, and InCites to retrieve the journal information and literature information of the selected journals and save them. Then, we logged into opencitations.net, entered the download page, selected the POCI data set, and downloaded the dump data in CSV format via the Figshare platform. Next, we used Navicat software (PremiumSoft CyberTech Ltd) to import all data into the local SQLite database and process the downloaded POCI data. Finally, in the local database based on the POCI data transformation, we used the PMID numbers extracted from the full records of the focus literature to identify the citation relationship associated with the focus papers. We established relevant data tables for the subsequent calculations.

The disruption index calculation was highly dependent on the time window chosen. The findings of Liu et al [27] confirm that the stabilization time window for the disruption index varies across different disciplines. We believe this is partly due to the different stages of development in various disciplines. Bornmann and Tekles [28] suggest that a time window of 3 years or more is an important prerequisite for ensuring the validity of the calculation results. Given the complexity of the disruption index calculation, the large volume of data [29], and our lack of access to extensive commercial citation data resources, we used the open citation data provided by OpenCitations for the calculation. To guarantee the accuracy of the results of the paper's innovation measurement while minimizing the computational workload, the citation time window in this study was set to 3 years.

Since Christensen [30] of Harvard Business School posited the theory of disruptive innovation in 1996, it has become an important paradigm in the field of innovation research. Based on this theory, Wu et al [31] put forward the disruption index (often referred to as the D Index for short), which can be used to measure disruption by calculating the citation substitution of focus papers in the citation network. Based on the disruption index, Bornmann et al [26] discussed the effectiveness of the convergence of this index and the variants that may improve the measurement effect. Ruan et al [32] analyzed the application limitations of the disruption index for measuring progress in the fields of science and technology.

Considering the process of metric calculation, F-type citations reflect that the focus paper disrupts the reference, B-type citations indicate that the focus paper is a development of the reference, and R-type citations reveal that the focus paper inherits the reference. Liu et al [33] argued that the disruption Index is a “relative” concept because it measures disruption based on the relative size of N_F (F-type citations) and N_B (B-type citations). They defined the disruption index, which is only reflected by F-type citations, as the “absolute disruption index.” Additionally, Liu et al [27,34] empirically studied the stabilization time window for the disruption index in different subject areas and resolved mathematical inconsistencies in its traditional formation (eg, N_R not satisfying monotonicity).

Based on this disruption index, Bornmann et al [35], Horen et al [36], Sullivan et al [37], Meyer et al [38], Jiang and Liu [39], and others have mined disruptive papers in the fields of scientometrics, craniofacial surgery, pediatric surgery, synthetic biology, and energy security respectively. On this basis, Jiang and Liu [40] proposed a method for calculating the Disruption index using open citation data during previous research. They carried out evaluations at both the literature and journal levels [41-45], contributing to a mature framework for evaluating disruptive innovation.

In equations (1) to (3), N_F refers to the literature that only cites the focus paper (FP); N_B refers to the literature that cites both the focus paper and at least one reference (R) of the focus paper; N_R refers to literature that only cites at least one reference (R) of the focus paper but not the focus paper; n is the number of articles contained in the journal; and D_Zi is the absolute disruption index (D_Z) of the i^th article in the journal.

Annual changes in JIF, JDI, and the number of published articles for the 7 journals selected in this study between 2017 and 2019 are shown in Table 1. The key findings are as follows. First, the average JDI of the selected authoritative journals is 0.5866, while the average JDI of OAMJs is 0.0255, indicating a significant difference. Second, the JDI values of the authoritative journals are higher than those of OAMJs. Third, JDI does not necessarily increase with a higher JIF.

Additionally, to better measure the difference in disruptive innovation level between OAMJ papers and authoritative journal papers, this study defined 2 parameters: the median D_Z of authoritative journals (MD_AJ) and the median D_Z of OAMJs (MD_OAMJ), which were 0.289 and 0.0019, respectively. The findings indicated that (1) the disruptive innovation level (94.60 %) of authoritative journal papers was higher than MD_OAMJ and (2) only 1.48% (861/58151) of OAMJ papers had a disruptive innovation level higher than MD_AJ. However, the number of OAMJ papers (n=861, 1.48%) was close to half of the number of authoritative journals (n=1128 of 2222).

Due to the complexity and diversity of clinical medical research, this study summarized statistics based on the classification results of InCites to assess the disruption of OAMJ papers and authoritative journal papers across various research fields. The findings, shown in Table 2, revealed several key insights.

There were differences in the average levels of disruptive innovation among OAMJ papers published in different research fields. The highest level was observed in the research field of immunology (0.1965), while neuroimaging had the lowest level (0.0049).

Similarly, there were differences in the average levels of disruptive innovation in authoritative journal papers, with reproductive biology showing the highest level (15.7902) and sport sciences (0.0031) showing the lowest level.

Across all fields, authoritative journal papers outperformed OAMJ papers in terms of disruptive innovation at the overall level. In 24 research directions such as allergy—accounting for 40.68% of all clinical medicine research areas—the number of OAMJ papers with disruptive innovation levels exceeding MD_AJ surpassed half the number of authoritative journal papers. Among the topics with no less than 10 authoritative papers published, the topics in which the number of papers published by OAMJ with a level of disruptive innovation exceeding MD_AJ's was no less than half of the number of papers published in authoritative journals accounted for 35.71% of the total.

The disruptive innovation levels of OAMJ papers and authoritative journal papers published by different countries are shown in Tables 3 and 4, which include data for countries with records in both types of journals. To better describe the relevant results, this study used the following criteria. First, the number of authoritative journal papers is used as a measure of national scientific research strength. A higher number of published authoritative papers reflects stronger national scientific research capabilities. Second, the difference in the disruptive innovation levels between published OAMJ papers and authoritative journal papers serves as a measure of the recognition of OAMJs by scholars in different countries. A smaller difference indicates a higher degree of recognition. Third, the ratio of OAMJ papers to authoritative journal papers measures the degree of favoritism scholars from different countries show toward OAMJs. A larger ratio suggests a higher degree of favoritism toward OAMJs by scholars from a particular country.

From Tables 3 and 4, we observed several key insights. First, there was a difference between the average disruptive innovation level of OAMJ papers across countries, with South Korea ranking the highest (0.5476) and Serbia ranking the lowest (0.0043). Similarly, the average disruptive innovation level of authoritative journal papers also varied across countries, with Peru ranking the highest (35.6125) and Serbia ranking the lowest (4.7261).

There was a linear relationship between the number of authoritative journal articles and the average disruptive innovation level of each country after logarithmization (as shown in Figure 1), with a correlation coefficient of –0.891, which indicated that authoritative journals did not consistently recognize the work of scholars from different countries. However, for OAMJ papers issued by each country, the average level of disruptive innovation did not show a linear relationship after logarithmic processing (as shown in Figure 2).

Scholars from different countries also had varying levels of recognition for OAMJs and tendencies to publish them. A higher recognition of OAMJs correlated with a higher likelihood of publishing papers in these journals (0.553). This was significantly correlated with the disruptive innovation levels of both authoritative journal papers (–0.461) and OAMJ papers (0.878). Moreover, the proportion of OAMJ papers meeting the MD_AJ threshold was significantly and positively correlated with the tendency to publish papers in OAMJs (0.689).

Among all countries, those that published more than 10% of OAMJ papers with a level of disruptive innovation above MD_AJ, while maintaining an average annual publication count of at least 1 in the selected journals, included Argentina, Turkey, Chile, Colombia, Pakistan, Philippines, Bangladesh, and Egypt.

This study has some limitations that must be acknowledged. First, participating authors with different ranks were treated as equal contributors, disregarding variations. Second, all focus papers were categorized under a single subject classification, without considering their inherent interdisciplinary properties. Third, only the knowledge diffusion of focus papers within the scope of periodical literature was considered, ignoring other venues. These limitations must be explored in future research based on natural language processing technology, and more accurate results can be obtained by jointly using multiple types of citation data sources.

The disruptive innovation levels of OAMJs were different from those of authoritative journals. However, OAMJs have become an important publication channel for innovative results in several research directions, providing fairer opportunities for researchers from limited-income countries to publish their work. Therefore, the academic community should acknowledge and appreciate the contributions of OAMJs to scientific research.