Background: In the current era of widespread access to the internet, we can monitor public interest in a topic via information-targeted web browsing. We sought to provide direct proof of the global population’s altered use of Wikipedia medical knowledge resulting from the new COVID-19 pandemic and related global restrictions.
Objective: We aimed to identify temporal search trends and quantify changes in access to Wikipedia Medicine Project articles that were related to the COVID-19 pandemic.
Methods: We performed a retrospective analysis of medical articles across nine language versions of Wikipedia and country-specific statistics for registered COVID-19 deaths. The observed patterns were compared to a forecast model of Wikipedia use, which was trained on data from 2015 to 2019. The model comprehensively analyzed specific articles and similarities between access count data from before (ie, several years prior) and during the COVID-19 pandemic. Wikipedia articles that were linked to those directly associated with the pandemic were evaluated in terms of degrees of separation and analyzed to identify similarities in access counts. We assessed the correlation between article access counts and the number of diagnosed COVID-19 cases and deaths to identify factors that drove interest in these articles and shifts in public interest during the subsequent phases of the pandemic.
Results: We observed a significant (P<.001) increase in the number of entries on Wikipedia medical articles during the pandemic period. The increased interest in COVID-19–related articles temporally correlated with the number of global COVID-19 deaths and consistently correlated with the number of region-specific COVID-19 deaths. Articles with low degrees of separation were significantly similar (P<.001) in terms of access patterns that were indicative of information-seeking patterns.
Conclusions: The analysis of Wikipedia medical article popularity could be a viable method for epidemiologic surveillance, as it provides important information about the reasons behind public attention and factors that sustain public interest in the long term. Moreover, Wikipedia users can potentially be directed to credible and valuable information sources that are linked with the most prominent articles.
After the COVID-19 pandemic outbreak began, a new concern for public health emerged—predicting and preventing the spread of the disease. The increased media coverage on the COVID-19 pandemic focused the public’s attention and likely affected the most popular internet search terms, thus altering people’s behavior worldwide [, ]. The public consumption of COVID-19 information in digital media is directly associated with preventive behaviors, including regularly washing hands with soap and water, staying away from crowded places, and wearing face masks in public [ ]. Bragazzi et al [ ] have suggested that internet search trend data can be used to build predictive models of disease spread and help with containing the pandemic. Bento et al [ ] has shown that the internet search term “coronavirus” increased in popularity immediately after the day of the first COVID-19 case announcement. However, the term’s popularity returned to the baseline level in less than 1-2 weeks. After this period, other terms that pertained to community-level policies (ie, quarantine, school closures, and COVID-19 tests) or personal health strategies (ie, masks, grocery delivery, and over-the-counter medications) emerged as the most-searched terms [ ]. Other studies have shown that public interest in specific search terms is more associated with reported deaths and media coverage than with real epidemiologic situations [ ]. Moreover, it has been observed that people quickly experience information overload, which results in information avoidance [ ]. Therefore, it seems logical to identify the most credible and reliable sources and use them to inform the public during the “information-hungry” period of any subsequent pandemics that may occur.
Wikipedia is considered a key web-based source of health information, and people are more willing to seek information that is published in Wikipedia than information from any other health websites . Wikipedia’s quality has been a highly debated topic, and many researchers are highly skeptical of the platform [ ]. Nevertheless, in 2005, a comparative study that was published in Nature reported that Wikipedia was competing head-to-head with Britannica [ ]. Since then, many different studies on Wikipedia's accuracy have shown that it is quite on par with published professional sources, as it provides reliable information [ , ]. Studies have reported that the quality of medical information that is available in Wikipedia is consistently high [ , ]. It has also been reported that Wikipedia is more reliable than several published sources, despite its low readability [ ]. Medical articles in Wikipedia are based on reliable sources and mainly skew toward the most prominent academic journals [ ]. More and more scholars have embraced the use of Wikipedia in the classroom [ , ]. The wide acceptance of Wikipedia is also related to institutional recognition. The American Psychological Association and the Association for Psychological Science have encouraged their members to edit Wikipedia articles [ , ].
With regard to medical information, Wikipedia's popularity exceeds that of the National Health Service, WebMD, Mayo Clinic, and World Health Organization (WHO) websites combined. This is mostly due to the highly accurate information that is provided by the editors of Wikiproject Medicine . Wikiprojects are edited by self-organized groups of volunteers who are involved in curating information on a specific topic [ ]. Since their reliability in providing medical information is so high, the WHO has partnered with the Wikimedia Foundation to expand access to trusted COVID-19–related information on Wikipedia [ ]. Several studies have indicated that medical students who use Wikipedia to prepare for their exams receive better grades than students who only rely on textbooks [ ]. Wikipedia has become a vital tool for global public health promotion [ ].
The quality and popularity of Wikipedia's medical content make analyzing the most popular medical articles extremely interesting from a medical professional’s point of view. Additionally, changes in viewership and peaks in interest are of high relevance to public health specialists. Given that Wikipedia has over 300 language variations that differ significantly in terms of rules, culture, and the presentation of knowledge , it is also interesting to compare changes in the popularity of Wikipedia medical articles across languages. Hence, to better understand public interest in high-quality health information during the COVID-19 pandemic, we conducted an analysis of daily visits to Wikipedia articles that were published from 2015 to 2020 and sought to identify possible factors for attracting public attention.
Detecting and Quantifying the Surge in COVID-19–Related Searches
We collected a list of 37,880 articles that were curated by the English Wikipedia Medicine Project. We derived the daily access counts (ie, from July 1, 2015, to September 13, 2020) of these articles by using ToolForge (ie, a pageview analytics tool). Daily access was defined as each visit to a given article on a given date. These data did not contain user-specific information.
We limited our selection to the 100 most accessed English Wikipedia Medicine Project articles that were published from July 1, 2015, to September 12, 2020. No other filters were applied to included Wikipedia articles. By using an interwiki mechanism (ie, cross-references for different language versions of Wikipedia), we identified articles that reported on the same subject in nine other language versions of Wikipedia (ie, French, German, Swedish, Dutch, Russian, Italian, Spanish, Polish, and Vietnamese). The articles were matched, and their daily access counts from July 1, 2015, to September 13, 2020 were obtained. These data were stored in an Excel sheet that used English Wikipedia article names. Extracted daily access data are provided in.
To provide additional context with regard to the ongoing COVID-19 pandemic, we also obtained pandemic-related data. We decided not to use reports on the total number of daily confirmed SARS-CoV-2 infection cases due to the possible influencing effect of introducing different public testing schemes. Other factors, such as the total number of daily deaths, are less likely to be altered by regional politics and health care discrepancies. Thus, we decided to measure the effect of the COVID-19 pandemic by analyzing the total number of global deaths and region-specific deaths, which were defined as the cumulative number of deaths resulting from SARS-CoV-2 infection for all reported countries and language-specific countries, respectively. These data were obtained from standardized public information in the Our World in Data University of Oxford initiative website .
The preprocessing of Wikipedia article access data was performed in Python 3.8, Excel version 2011 (Microsoft Corporation). Due to the influences of day-to-day differences in Wikipedia access and absolute differences in access to the different language versions of Wikipedia, we standardized access data by calculating daily article access as percentages of language-specific access to Wikipedia on a given day.
To identify deviations in the stability of article visits, we chose articles that exhibited the least altered access patterns throughout the investigated period. Articles were selected based on their relative stability (ie, the SD of percent article access in a 30-day moving window divided by the 30-day moving average [ie, mean] of percent article access), which was calculated for the full duration of investigated period. Relative stability was assessed to identify articles that exhibited relatively unchanged access patterns throughout the studied period. Reference articles were defined as the 20 most stable articles across all languages. These articles provided the highest mean daily access percentages for all evaluated periods. As such, a reference article could be used for direct comparisons with other highly accessed articles. They also provided metrics that were the least affected by changes in Wikipedia use.
A statistical analysis was conducted to identify access patterns in Wikipedia and Wikipedia Medicine Project articles that were published in 2020 and to determine these patterns’ association with the COVID-19 pandemic. First, we investigated the association between Wikipedia access before and during the COVID-19 pandemic. Second, we determined whether there were specific articles of interest before and during the COVID-19 pandemic (ie, excluding the years that were associated with other epidemics that were covered in media). We also investigated whether the total number of global and regional deaths resulting from SARS-CoV-2 were associated with increased access to articles of interest during the COVID-19 pandemic.
We also investigated whether there was a difference between navigation to Wikipedia articles of interest before and during the COVID-19 pandemic. To this end, we determined the minimum number of links required to navigate between two articles within Wikipedia (ie, the degree of separation [DOS]).
The statistical analysis was conducted in Python 3.8 and Statistica 13.3 (Statsoft, TIBCO Software Inc, Dell Inc). The DOS was assessed by using the PHP (hypertext preprocessor) language and jQuery library, which were implemented in the Degrees of Wikipedia tool . To facilitate the straightforward representation of DOS results, we provided the DOSs of articles of interest that were found across the highest number of Wikipedia language versions. Due to the observed lack of a normal distribution in article access, we used nonparametric methods. We considered a P value of <.05 to be statistically significant.
Abnormalities in Wikipedia access patterns that occurred from July 1, 2015, to September 13, 2020, were investigated by using the Chi-square goodness-of-fit test. To identify the qualitative changes in the annual top 10 most accessed Wikipedia Medicine Project articles, we used word clouds as graphical representations of access patterns.
Identifying Shifts in Search Patterns Before and During the COVID-19 Pandemic
We determined whether the COVID-19 pandemic altered temporal article access patterns. This was achieved by using a k-nearest neighbor (kNN) unsupervised clustering method for analyzing selected Wikipedia language versions. As we were interested in the patterns of access to articles that were of increased user interest in a given period, we performed separate unsupervised clustering analyses for the 2017-2019 and 2020 periods. We excluded the 2015-2016 period due to the possible influence of the Zika virus epidemic, which could have promoted interests in the general population that were similar to interests during the COVID-19 pandemic. As such, we were able to identify articles of user interest during the 2017-2019 and 2020 periods.
We provided a tabular representation of kNN-recognized articles that were associated with increased access before and during the COVID-19 pandemic. The weights that were detected by the kNN algorithm were used to analyze monthly changes in the public’s interest in overrepresented articles during the COVID-19 pandemic and these articles’ relation to the total number of global and regional deaths resulting from SARS-CoV-2 infection. This correlation was tested by performing a Spearman rank correlation analysis. We further divided the data by month (ie, March to September 2020) to evaluate whether the correlation changed between months.
We compared navigation patterns between articles of interest that were published before and during the COVID-19 pandemic by using the DOSs of reference articles. These patterns were tested with the Kruskall-Wallis test, Dunn posthoc test, and U Mann-Whitney test.
Characteristics of Wikipedia Medicine Project Articles
We analyzed visits to Wikipedia Medicine Project articles that were published in 10 language versions of Wikipedia from 2015 to 2020. These languages included English, French, German, Swedish, Dutch, Russian, Italian, Spanish, Polish, and Vietnamese. We analyzed 100 articles from English Wikipedia after mapping the other language versions of Wikipedia (ie, 100 articles from French Wikipedia, 98 from Dutch Wikipedia, 98 from Spanish Wikipedia, 97 from Russian Wikipedia, 96 from Italian Wikipedia, 95 from German Wikipedia, 94 from Polish Wikipedia, and 93 from Vietnamese Wikipedia). Due to its low coverage (ie, <90 matched articles, as determined via the interwiki mechanism), Swedish Wikipedia was removed from the analysis. Wikipedia articles without coverage across all languages included the following: “Bed bug” (25% coverage), “Coronavirus” (50% coverage), “Adderall” (62.5% coverage), “Bubonic plague” (62.5% coverage), “Chronic traumatic encephalopathy” (62.5% coverage); “Plantar fasciitis” (75% coverage), “Alprazolam” (87.5% coverage), “Cancer” (87.5% coverage), “Elizabeth Holmes” (87.5% coverage), “Escitalopram” (87.5% coverage), “Ketogenic diet” (87.5% coverage), “Lorazepam” (87.5% coverage), “Project MKUltra” (87.5% coverage), “Psychopathy” (87.5% coverage), and “Trypophobia” (87.5% coverage). Full lists of articles that are available for each language are provided in, Supplementary Table S1.
In the 2015-2020 period, there were a total of 2,258,621,012 Wikipedia entries in the top 100 most accessed articles and 775,791,941,485 overall visits to Wikipedia across all selected languages (Wikipedia access across all languages: median 0.41%; range 0.29%-0.46%). Annual access to Wikipedia significantly and gradually decreased from 2015 to 2020 (r=−0.0513; P=.03). Significant increases in use (P<.001) were observed for the English (r=0.0619), Polish (r=0.1237), Dutch (r=0.0481), Italian (r=0.2197), and Vietnamese (r=0.7480) versions of Wikipedia. Significant decreases in use (P<.001) were observed for the German (r=−0.2654), Spanish (r=−0.1085), and Russian (r=−0.4338) versions of Wikipedia. For the top 100 most accessed Wikipedia medical articles, we observed a 1.5-fold to 3.8-fold increase in access across all language versions of Wikipedia in 2020 (ie, compared to access in previous years).
Surge in Article Visits During Early 2020
To determine the effect of the COVID-19 pandemic on Wikipedia use, we used a graphical representation of total daily Wikipedia access (). Due to the suspected increase in Wikipedia use during 2020, we constructed a generalized additive model by using a limited-memory Broyden-Fletcher-Goldfarb-Shanno fitting algorithm that was implemented in the FBProphet tool to predict Wikipedia use in 2020 based on previous access patterns. We based the model on data from the 2015-2019 period to identify expected behaviors in 2020 and compare them to observed access patterns. The global pattern for Wikipedia access was indicative of seasonal behaviors ( ). This pattern changed across all language versions of Wikipedia in March 2020, and it returned to normal in September 2020, as predicted by our models. The Chi-square goodness-of-fit test values for monthly Wikipedia access had P values of <.001 across all eight languages ( ).
To determine how abnormal Wikipedia access altered the relative search stability of articles, we needed to pick a point of reference. We determined that the “Sexual intercourse” article demonstrated high yet stable monthly access within the evaluated period. By comparing the “Sexual intercourse” article to the “Spanish flu” article, we were able to determine that the relative access to these articles before the COVID-19 pandemic changed during the pandemic (“Spanish flu” article: P<.04; “Sexual intercourse” article: P<.001;).
Effects of the COVID-19 Pandemic on the Annual Top 10 Most Commonly Accessed Articles in Wikipedia
We provided a graphical representation of the annual top 10 most accessed articles by creating word clouds for articles that were published from 2015 to 2020 (, , Supplementary Figures S10-S17). English Wikipedia word clouds are shown in , and those for other language versions of Wikipedia are shown in , Supplementary Figures S10-S17. We investigated how the 10 most viewed articles changed each year. In the 2015-2019 period, we observed a set of articles that were constantly among the top 10 most accessed articles each year. These articles were as follows: “Leonardo da Vinci” (appearance frequency: 37/45), “Asperger syndrome” (appearance frequency: 36/45), and “Bipolar disorder” (appearance frequency: 28/45). Compared to previous years, there was a distinctive difference in the top 10 most accessed articles in 2020; the “Pandemic” (appearance frequency: 9/9), “COVID-19 pandemic” (appearance frequency: 9/9), “Coronavirus disease 2019” (appearance frequency: 8/9), and “Spanish flu” (appearance frequency: 8/9) articles became the most frequently accessed articles. We found 8 new articles among the top 10 most accessed articles in 2020 that have never appeared in this list. Of these articles, 2 were created after the SARS-CoV-2 pandemic (ie, the “COVID-19 pandemic” and “Coronavirus disease 2019” articles), while the other 6 were present within Wikipedia before the pandemic (ie, the ”Pandemic,” “Spanish flu,” “Coronavirus,” “Bubonic plague,” ”Influenza,” and “World Health Organization” articles).
Identifying Cross-Language Similarities in Article Access Prior to and During the COVID-19 Pandemic via Unsupervised Clustering
The across-language abnormality that was observed in Wikipedia and Wikipedia Medicine Project access frequency in 2020 was further investigated. By conducting an unsupervised analysis, we identified differences between article access before and during the COVID-19 pandemic. Articles that were published in 2020 and recognized by the kNN algorithm were considered COVID-19–related articles. These included the following eight articles: “Coronavirus,” “Spanish flu,” “Coronavirus disease 2019,” “COVID-19 pandemic, “Influenza,” “Pandemic,” “Virus,” and “Black Death.” The group of articles that were unrelated to COVID-19 included 25 articles. We selected the four articles from this group that were present in most language versions of Wikipedia (ie, the “Sexual intercourse,” “Leonardo da Vinci,” “Bipolar disorder,” and “Borderline personality disorder” articles).
We compared the DOSs of COVID-19–related articles to the DOSs of articles that were unrelated to COVID-19. We used the “COVID-19 pandemic” article as a point of reference for COVID-19–related articles and the “Sexual intercourse” article as a point of reference for articles unrelated to COVID-19. The analysis showed that the DOSs between articles unrelated to COVID-19 and the “Sexual intercourse” article were significantly higher than the DOSs between COVID-19–related articles and the “COVID-19 pandemic” article (Kruskall-Wallis test and posthoc Dunn test: P<.001;). This indicated that COVID-19–related articles were more closely connected, which we suspected due to the observed similarities in themes.
We performed a Spearman rank correlation analysis on articles to provide additional context. We observed that articles unrelated to COVID-19 had higher median correlation values across languages than those in the COVID-19–related article group (). In the COVID-19–related article group, the highest median correlation value (ie, across all languages) was found between the “COVID-19 pandemic” and “Coronavirus disease 2019” articles (R=0.7022), and the lowest median correlation value was found between the “COVID-19 pandemic” and “Influenza” articles (R=0.0330). In the group of articles unrelated to COVID-19, correlation values were higher. The highest median correlation value was found between the “Sexual intercourse” and “Bipolar disorder” articles (R=0.8657), and the lowest median correlation value was found between the “Sexual intercourse” and “Leonardo da Vinci” articles (R=0.4631).
We also compared the DOSs between all articles that were analyzed in detail and the “COVID-19 pandemic” article. COVID-19–related articles had significantly lower DOSs compared to articles unrelated to COVID-19 (U Mann-Whitney test: P<.001;).
Finally, we investigated how the patterns in articles access (ie, those identified by unsupervised clustering) were associated with the total number of global and region-specific deaths resulting from SARS-CoV-2 infection. To this end, we standardized each measure. We used ordinary least squares linear regression to identify correlations across each month in the March to September 2020 period (, , Supplementary Table S3).
Upon further investigation, we found that the kNN-derived pattern in COVID-19–related articles’ access was significantly associated with both the total number of global and region-specific deaths resulting from SARS-CoV-2 infection for articles across all Wikipedia language versions (). There was a notable difference in the correlation between article access and the total number of deaths resulting from SARS-CoV-2 infection, which appeared linear for region-specific deaths and negatively exponential for global deaths ( ). This was also reflected by the low absolute Spearman rank coefficients between article access and the total number of global deaths resulting from SARS-CoV-2 infection for the months after June 2020 ( ).
|Wikipedia articles||Prevalencea, %||Spearman Rb (within groups)||DOSs within groupsc (IQR)||DOSs across articlesd (IQR)|
|COVID-19–related article group|
|Coronavirus||80||0.2770||1 (1-1)||1 (1-1)|
|Spanish flu||55.6||0.4451||2 (1.75-2)||2 (1.75-2)|
|Coronavirus disease 2019||55.6||0.7022||1 (1-1.25)||1 (1-1.25)|
|Pandemic||44.4||0.4341||1 (1-1)||1 (1-1)|
|Black death||33.3||0.4332||2 (2-2)||2 (2-2)|
|Virus||11.1||0.3851||2 (1-2)||2 (1-2)|
|Influenza||11.1||0.0330||1.5 (1-2)||1.5 (1-2)|
|Group of articles unrelated to COVID-19|
|Sexual intercourse||88.9||Referencee||Referencee||3 (2-3)|
|Leonardo da Vinci||88.9||0.4631||2 (1.5-2)||2 (2-2.5)|
|Bipolar disorder||77.8||0.8657||3 (2-3)||3 (2.75-3)|
|Borderline personality disorder||77.8||0.8352||2.5 (2-3)||3 (3-3)|
aRefers to the presence of an article in k-nearest neighbor–determined groups across all available languages.
bThe Spearman R value for an article.
cThe DOS of an article within a k-nearest neighbor–determined group.
dThe DOS across all articles that were investigated in detail.
eThe article of reference for calculating DOSs.
Our study shows that the pandemic has significantly influenced patterns in Wikipedia access. There was an apparent surge in interest for infectious disease, which somewhat surprisingly and quickly declined despite the mounting death toll. This is the first study on the impact of the previously recognized change in society’s interest, which resulted from the COVID-19 pandemic. In this study, this impact was reflected by changes in Wikipedia access patterns. Moreover, the analysis of different Wikipedia language versions allowed us to confirm that the observed effect of these changes is reflected by both the regional and global impacts of the COVID-19 pandemic. Our study provides proof that an unsupervised clustering method for analyzing data on daily access to medical information could be used to identify interests in global health issues.
Recent studies have indicated that the global lockdown is a possible reason for the change in internet use . We noticed a pronounced increase in the total number of visits to Wikipedia in March 2020. Not only did we observe more frequent visits to Wikipedia, but we also observed a distinctive change in searched topics. The list of the top 10 most accessed articles in 2020 included articles that have not appeared in such lists during previous years. Moreover, the DOS between most of the articles of interest was low, suggesting that navigation was relevant to the observed change. This finding, combined with the correlations between article access, indicated that during the pandemic, people had an increased interest in topics that were not directly related to COVID-19 but were related to pandemics in general. Correlations in access to the “Spanish flu,” “Black Death,” and “Bubonic plague” articles show that the general public has rapidly gained interest in the topic of previous infectious disease–related health crises. We suspect that these observed correlations are associated with the COVID-19 Wikiproject, which curates articles to provide users with valuable information. The COVID-19 Wikiproject, which is managed by 1200 editors, has resulted in the addition of more than 6500 entries to Wikipedia [ ].
Our study shows that similar changes in public interests have already occurred in recent years. We noticed an increase in access to the 2016 “Zika virus” article in 5 of the 9 investigated language versions of Wikipedia. This could be associated with the epidemic that was announced by the WHO in November 2016. This association was also confirmed in the Bragazzi et al  and Hickmann et al [ ] studies on H1N1-related and Zika virus–related articles and these viruses’ respective outbreaks. Other studies have focused on the increased amount of searches for terms related to anosmia (ie, a characteristic symptom of COVID-19) and have shown its correlation with the SARS-CoV-2 outbreak in many countries [ ]. Moreover, trends in the amount of searches for the terms “wash hands,” “hand sanitizer,” and “antiseptic” have successfully predicted the rising number of COVID-19 cases in many countries, which indicates that search trends can potentially be used in epidemiologic surveillance [ ]. In our analysis, we noticed an increase in the popularity of a Wikipedia article about the WHO. This result may indicate that the general public has decided to learn more about the organization, making the WHO a focal point for the outbranching of searches. Conversely, this increase in popularity could be linked with President Donald Trump’s attempts to discredit the WHO for the way that they handled the COVID-19 outbreak. These attempts were highly amplified by mainstream media and Twitter [ ].
We were interested in identifying the aspect of the pandemic that specifically drove people’s growing interest in COVID-19 from March to June 2020. Our unsupervised clustering analysis showed that the total number of global deaths significantly correlated with the temporal increase in the number of searches for COVID-19–related articles (ie, from March to June 2020). Interestingly, despite the continuous rise in the number of global deaths, the number of visits to COVID-19–related articles decreased after June 2020. This could be interpreted as a gradual decline in global interest toward the pandemic. Data on interest in COVID-19–related articles suggest that there is an increased interest in contagious diseases and that an opportunity to raise awareness through Wikipedia-centered strategies will arise during future outbreaks of infectious diseases. However, the stable correlation between the region-specific number of deaths and the frequency of searches for COVID-19–related articles that persisted until September 2020 is a noteworthy finding. A study by Gozii et al  showed that public attention and response are mostly driven by media coverage instead of disease spread. Interestingly, they also showed that the media sharply shifted their focus toward domestic situations as soon as the first death was confirmed in a person’s home country [ ]. Therefore, we conclude that reporting about region-specific death reshapes individuals’ perceptions of risk and significantly impacts public interest, thereby affecting the information-seeking behaviors of people from affected areas.
Our study's limitations include the fact that our analysis was restricted to article access data, which did not contain detailed information on user-specific article access patterns or users’ demographic characteristics (eg, age, gender, and educational status). However, we refrained from performing such analyses to maintain the privacy of Wikipedia users. Our study proves that it is possible to identify the global and regional effects of a health crisis on people's behavior. The in-depth analysis of users’ behaviors can be unethical due to its potential for targeting region-specific populations and spreading disinformation, which are ever-present concerns due to the malicious spread of fake news.
Language bias may have also slightly skewed our results, as preferences for English may have resulted in an underreporting of native language–specific searches. Moreover, several idiomatic phrases that are specific to the English vocabulary—“black death” being the prime example—may not directly translate to other languages. To mitigate this bias, we assumed that analyzing data from English Wikipedia would account for most of the global population’s language preferences. Further, we used search data from the other language versions of Wikipedia to validate our results. This strategy yielded a cohesive and surprisingly uniform sample of COVID-19–related search data.
Our results support the idea that Wikipedia can be used as a tool for successfully surveilling trends in public interest. The increase in interest toward COVID-19–related articles was followed by a progressive decline. This shows that the potential optimal window for efficient information dissemination via Wikipedia is the early phase of a pandemic. Wikipedia articles that directly link to major articles about a major health crisis can contribute to the spread of global anxiety and the promotion of prevention behaviors. Therefore, Wikipedia articles should be carefully selected.
DJ’s participation was possible thanks to a grant (grant number: 2019/35/B/HS6/01056) from the Polish National Science Centre.
Conflicts of Interest
Author DJ is a non-paid, volunteer member of the Board of Trustees of Wikimedia Foundation, a non-profit publisher of Wikipedia. All the other authors have no conflicts to declare.
Open data.XLSX File (Microsoft Excel File), 22109 KB
Supplementary tables S1-S3.XLSX File (Microsoft Excel File), 70 KB
Word cloud.PDF File (Adobe PDF File), 327 KB
Supplementary figures S1-S25.PDF File (Adobe PDF File), 1291 KB
- Ashrafi-Rizi H, Kazempour Z. Information typology in coronavirus (COVID-19) crisis; a commentary. Arch Acad Emerg Med 2020 Mar 12;8(1):e19 [FREE Full text] [Medline]
- Ashrafi-Rizi H, Kazempour Z. Information diet in Covid-19 crisis; a commentary. Arch Acad Emerg Med 2020 Mar 28;8(1):e30 [FREE Full text] [Medline]
- Liu PL. COVID-19 information seeking on digital media and preventive behaviors: The mediation role of worry. Cyberpsychol Behav Soc Netw 2020 Oct;23(10):677-682. [CrossRef] [Medline]
- Bragazzi NL, Dai H, Damiani G, Behzadifar M, Martini M, Wu J. How big data and artificial intelligence can help better manage the COVID-19 pandemic. Int J Environ Res Public Health 2020 May 02;17(9):3176 [FREE Full text] [CrossRef] [Medline]
- Bento AI, Nguyen T, Wing C, Lozano-Rojas F, Ahn YY, Simon K. Evidence from internet search data shows information-seeking responses to news of local COVID-19 cases. Proc Natl Acad Sci U S A 2020 May 26;117(21):11220-11222 [FREE Full text] [CrossRef] [Medline]
- Sousa-Pinto B, Anto A, Czarlewski W, Anto JM, Fonseca JA, Bousquet J. Assessment of the impact of media coverage on COVID-19-related Google Trends data: Infodemiology study. J Med Internet Res 2020 Aug 10;22(8):e19611 [FREE Full text] [CrossRef] [Medline]
- Soroya SH, Farooq A, Mahmood K, Isoaho J, Zara SE. From information seeking to information avoidance: Understanding the health information behavior during a global health crisis. Inf Process Manag 2021 Mar;58(2):102440 [FREE Full text] [CrossRef] [Medline]
- Laurent MR, Vickers TJ. Seeking health information online: does Wikipedia matter? J Am Med Inform Assoc 2009;16(4):471-479 [FREE Full text] [CrossRef] [Medline]
- Jemielniak D. Wikipedia: Why is the common knowledge resource still neglected by academics? Gigascience 2019 Dec 01;8(12):giz139 [FREE Full text] [CrossRef] [Medline]
- Giles J. Internet encyclopaedias go head to head. Nature 2005 Dec 15;438(7070):900-901. [CrossRef] [Medline]
- Dunn PK, Marshman M, McDougall R. Evaluating Wikipedia as a self-learning resource for statistics: You know they'll use it. Am Stat 2018 Jun 11;73(3):224-231. [CrossRef]
- Heilman JM, Kemmann E, Bonert M, Chatterjee A, Ragar B, Beards GM, et al. Wikipedia: a key tool for global public health promotion. J Med Internet Res 2011 Jan 31;13(1):e14 [FREE Full text] [CrossRef] [Medline]
- Mesgari M, Okoli C, Mehdi M, Nielsen FÅ, Lanamäki A. “The sum of all human knowledge”: A systematic review of scholarly research on the content of Wikipedia. J Assoc Inf Sci Technol 2014 Dec 02;66(2):219-245. [CrossRef]
- Shafee T, Masukume G, Kipersztok L, Das D, Häggström M, Heilman J. Evolution of Wikipedia's medical content: past, present and future. J Epidemiol Community Health 2017 Nov;71(11):1122-1129 [FREE Full text] [CrossRef] [Medline]
- Matheson D, Matheson-Monnet C. Wikipedia as informal self-education for clinical decision-making in medical practice. Open Med J 2017 Sep 30;4(1):15-25 [FREE Full text] [CrossRef]
- Jemielniak D, Masukume G, Wilamowski M. The most influential medical journals according to Wikipedia: Quantitative analysis. J Med Internet Res 2019 Jan 18;21(1):e11429 [FREE Full text] [CrossRef] [Medline]
- Konieczny P. Teaching with Wikipedia in a 21st‐century classroom: Perceptions of Wikipedia and its educational benefits. J Assoc Inf Sci Technol 2016 Mar 15;67(7):1523-1534. [CrossRef]
- Jemielniak D, Aibar E. Bridging the gap between wikipedia and academia. J Assoc Inf Sci Technol 2016 Apr 04;67(7):1773-1776. [CrossRef]
- Breckler S. Don’t like Wikipedia? Change it. American Psychological Association. 2010 Dec. URL: https://www.apa.org/science/about/psa/2010/12/wikipedia-change [accessed 2020-12-06]
- Mahzarin B. Harnessing the power of Wikipedia for scientific psychology: A Call to Action. Association for Psychological Science. 2011 Feb 11. URL: https://www.psychologicalscience.org/observer/harnessing-the-power-of-wikipedia-for-scientific-psychology-a-call-to-action [accessed 2020-12-06]
- James R. WikiProject Medicine: creating credibility in consumer health. J Hosp Librariansh 2016 Oct 18;16(4):344-351. [CrossRef]
- The World Health Organization and Wikimedia Foundation expand access to trusted information about COVID-19 on Wikipedia. World Health Organization. 2020 Oct 22. URL: https://www.who.int/news/item/22-10-2020-the-world-health-organization-and-wikimedia-foundation-expand-access-to-trusted-information-about-covid-19-on-wikipedia [accessed 2020-12-06]
- Azzam A, Bresler D, Leon A, Maggio L, Whitaker E, Heilman J, et al. Why medical schools should embrace Wikipedia: Final-year medical student contributions to Wikipedia articles for academic credit at one school. Acad Med 2017 Feb;92(2):194-200 [FREE Full text] [CrossRef] [Medline]
- Jemielniak D, Wilamowski M. Cultural diversity of quality of information on Wikipedias. J Assoc Inf Sci Technol 2017 Aug 07;68(10):2460-2470. [CrossRef]
- Reavley NJ, Mackinnon AJ, Morgan AJ, Alvarez-Jimenez M, Hetrick SE, Killackey E, et al. Quality of information sources about mental disorders: a comparison of Wikipedia with centrally controlled web and printed sources. Psychol Med 2012 Aug;42(8):1753-1762. [CrossRef] [Medline]
- Coronavirus pandemic (COVID-19) - statistics and research. Our World in Data. URL: https://ourworldindata.org/coronavirus [accessed 2020-09-17]
- Link two articles on Wikipedia. Degrees of Wikipedia. URL: https://degreesofwikipedia.com/ [accessed 2021-03-10]
- Brodeur A, Clark AE, Fleche S, Powdthavee N. COVID-19, lockdowns and well-being: Evidence from Google Trends. J Public Econ 2021 Jan;193:104346 [FREE Full text] [CrossRef] [Medline]
- Benjakob O. Why Wikipedia is immune to coronavirus. Haaretz. 2020 Apr 08. URL: https://www.haaretz.com/us-news/.premium.MAGAZINE-why-wikipedia-is-immune-to-coronavirus-1.8751147 [accessed 2020-12-06]
- Bragazzi NL, Alicino C, Trucchi C, Paganino C, Barberis I, Martini M, et al. Global reaction to the recent outbreaks of Zika virus: Insights from a Big Data analysis. PLoS One 2017 Sep 21;12(9):e0185263. [CrossRef] [Medline]
- Hickmann KS, Fairchild G, Priedhorsky R, Generous N, Hyman JM, Deshpande A, et al. Forecasting the 2013-2014 influenza season using Wikipedia. PLoS Comput Biol 2015 May 14;11(5):e1004239. [CrossRef] [Medline]
- Walker A, Hopkins C, Surda P. Use of Google Trends to investigate loss-of-smell-related searches during the COVID-19 outbreak. Int Forum Allergy Rhinol 2020 Jul;10(7):839-847 [FREE Full text] [CrossRef] [Medline]
- Lin YH, Liu CH, Chiu YC. Google searches for the keywords of "wash hands" predict the speed of national spread of COVID-19 outbreak among 21 countries. Brain Behav Immun 2020 Jul;87:30-32 [FREE Full text] [CrossRef] [Medline]
- Gramer R, Lynch C. Trump seeks to halve U.S. funding for World Health Organization as coronavirus rages. Foreign Policy. 2020 Feb 10. URL: https://foreignpolicy.com/2020/02/10/trump-world-health-organization-funding-coronavirus-state-department-usaid-budget-cuts/ [accessed 2021-03-16]
- Gozzi N, Tizzani M, Starnini M, Ciulla F, Paolotti D, Panisson A, et al. Collective response to media coverage of the COVID-19 pandemic on Reddit and Wikipedia: Mixed-methods analysis. J Med Internet Res 2020 Oct 12;22(10):e21597 [FREE Full text] [CrossRef] [Medline]
|DOS: degree of separation|
|kNN: k-nearest neighbor|
|PHP: hypertext preprocessor|
|WHO: World Health Organization|
Edited by R Kukafka, C Basch; submitted 07.12.20; peer-reviewed by M Martini, E Marzilli; comments to author 12.01.21; revised version received 21.01.21; accepted 18.02.21; published 12.04.21Copyright
©Jędrzej Chrzanowski, Julia Sołek, Wojciech Fendler, Dariusz Jemielniak. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 12.04.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.