The NICE search filters for treating and managing COVID-19: validation in MEDLINE and Embase (Ovid)

Abstract

Objective: In this paper we report how the United Kingdom's National Institute for Health and Care Excellence (NICE) search filters for treating and managing COVID-19 were validated for use in MEDLINE (Ovid) and Embase (Ovid). The objective was to achieve at least 98.9% for recall and 64% for precision. Methods: We did two tests of recall to finalize the draft search filters. We updated the data from an earlier peer-reviewed publication for the first recall test. For the second test, we collated a set of systematic reviews from Epistemonikos COVID-19 L.OVE and extracted their primary studies. We calculated precision by screening all the results retrieved by the draft search filters from a targeted sample covering 2020-23. We developed a gold-standard set to validate the search filter by using all articles available from the "Treatment and Management" subject filter in the Cochrane COVID-19 Study Register. Results: In the first recall test, both filters had 99.5% recall. In the second test, recall was 99.7% and 99.8% in MEDLINE and Embase respectively. Precision was 91.1% in a deduplicated sample of records. In validation, we found the MEDLINE filter had recall of 99.86% of the 14,625 records in the gold-standard set. The Embase filter had 99.88% recall of 19,371 records. Conclusion: We have validated search filters to identify records on treating and managing COVID-19. The filters may require subsequent updates, if new SARS-CoV-2 variants of concern or interest are discussed in future literature.