Published on in Vol 23, No 5 (2021): May

Preprints (earlier versions) of this paper are available at, first published .
Authors’ Reply to: Redundancy of Terms in Search Strategies. Comment on “Searching PubMed to Retrieve Publications on the COVID-19 Pandemic: Comparative Analysis of Search Strings”

Authors’ Reply to: Redundancy of Terms in Search Strategies. Comment on “Searching PubMed to Retrieve Publications on the COVID-19 Pandemic: Comparative Analysis of Search Strings”

Authors’ Reply to: Redundancy of Terms in Search Strategies. Comment on “Searching PubMed to Retrieve Publications on the COVID-19 Pandemic: Comparative Analysis of Search Strings”

Letter to the Editor

1Danish Diabetes Knowledge Center, Steno Diabetes Center Copenhagen, Gentofte, Denmark

2Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, QC, Canada

3NYU Langone Health, NYU Grossman School of Medicine, NYU Health Sciences Library, New York, NY, United States

4Barcelona Institute for Global Health (ISGlobal), Hospital Clínic, University of Barcelona, Barcelona, Spain

Corresponding Author:

Jeffrey V Lazarus, PhD

Barcelona Institute for Global Health (ISGlobal), Hospital Clínic

University of Barcelona

Calle del Rosellón 132

Barcelona, 08036



We appreciate the interest in our analyses and dedication to informing clinicians, reviewers, information specialists, and others about searching for articles in PubMed.

The letter authors [1] highlight the importance of avoiding search errors and the principle of parsimony in formulating search strings, encouraging searchers “to eliminate any terms or phrases from a search strategy that do not retrieve or provide new records, as they are thus unnecessary.” We support the ambition and principle of parsimony, agreeing that redundant terms should be avoided. In our view, search 1 contains no redundant terms [2]. It is unclear to us what search the letter authors refer to, and we encourage them to specify their critique regarding this. To avoid misunderstandings, the example search string in the letter does not originate from any that we tested.

Our analysis showed how term choice and combinations impacted the sensitivity, precision, and F-score of selected search strings in PubMed in the 10 weeks from when the World Health Organization declared COVID-19 a public health emergency of international concern. Our results demonstrated that the more elaborate searches 1 and 2 had higher sensitivity than the ones singled out in the letter (searches 4 and 5). Please note that searches 3 and 4 were in practice identical, which some readers may overlook. In both the abstract and article, we highlighted the value of applying the single-term search “COVID-19” (searches 3 and 4) for everyday searches. However, as reflected by the F-score, searches 1 and 2 performed best.

We acknowledge that the differences between some of the analyzed search strings are minor to some PubMed users and for some search purposes. For Cochrane-style systematic reviews, we recommend the more comprehensive search strings with the higher sensitivity and specificity unless resources are scarce. For everyday informational needs, less comprehensive searches may suffice.

Our study had limitations, importantly the timeframe represented. As COVID-19 has evolved (eg, variants have been detected and have spread), new terms may be pertinent to add to search strings, including relevant Medical Subject Headings (MeSH) terms, which were not available during our search. Instead, we used all available Supplementary Concepts identified as relevant to COVID-19 in June 2020. Since the beginning of 2021, COVID-19–relevant MeSH terms have been available [3], which we would include if we were developing a comprehensive search string now.

We note that the suggestion from the letter authors to include the controlled vocabulary term “COVID-19” (unique ID: C000657245) would not activate the MeSH term “COVID-19” (unique ID: D000086382). Instead, it activates the obsolete Supplementary Concept “COVID-19”, which we included in our search string.

The change in availability of relevant controlled vocabulary terms highlights the need to quickly add additional features, enabling PubMed users to perform effective searches during a pandemic. In line with our suggestion that the National Library of Medicine add a new subject filter, such as covid-19[sb], to address this issue, they have recently added several COVID-19 filters, such as LitCGeneral[filter], that may become useful [4]. For future pandemics, we hope that such features would be made available sooner and that diseases and their variants would be named earlier.

Conflicts of Interest

LNR, ON, and THA are employed at Steno Diabetes Center Copenhagen, a public hospital and research institution under the Capital Region of Denmark, which is partly funded by a grant from the Novo Nordisk Foundation. The funders had no role in this work. There are no other conflicts to declare.


  1. Campos DMDO, Fulco UL, Oliveira JIN. Redundancy of terms in search strategies to identify COVID-19–related articles in PubMed: comment on “Searching pubmed to Retrieve Publications on the COVID-19 Pandemic: Comparative Analysis of Search Strings”. J Med Internet Res 2021 May 28;23(5):e28666 [FREE Full text] [CrossRef] [Medline]
  2. Lazarus JV, Palayew A, Rasmussen LN, Andersen TH, Nicholson J, Norgaard O. Searching PubMed to retrieve publications on the COVID-19 pandemic: comparative analysis of search strings. J Med Internet Res 2020 Nov 26;22(11):e23449 [FREE Full text] [CrossRef] [Medline]
  3. Tybaert S. MEDLINE data changes—2021. NLM Tech Bull 2020;437:e3a [FREE Full text]
  4. PubMed user guide - COVID-19 article filters. National Library of Medicine.   URL: [accessed 2020-04-08]

Edited by T Derrick; This is a non–peer-reviewed article. submitted 09.04.21; accepted 13.05.21; published 28.05.21


©Lauge Neimann Rasmussen, Ole Norgaard, Tue Helms Andersen, Adam Palayew, Joey Nicholson, Jeffrey V Lazarus. Originally published in the Journal of Medical Internet Research (, 28.05.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.