Mapping the content of comments on bioRxiv and medRxiv preprints

Abstract

AbstractIntroductionPreprints have been increasingly used in biomedical sciences, providing the opportunity for research to be publicly assessed before journal publication. With the increase in attention over preprints during the COVID-19 pandemic, we decided to assess the content of comments left on preprint platforms.MethodsPreprints posted on bioRxiv and medRxiv in 2020 were accessed through each platform’s API, and a random sample of preprints that had received between 1 and 20 comments was analyzed. Comments were evaluated in triplicate by independent evaluators using an instrument that assessed their features and general content.Results7.3% of preprints received at least 1 comment during a mean follow-up of 7.5 months. Analyzed comments had a median size of 43 words. Criticisms, corrections or suggestions were the most prevalent type of content, followed by compliments or positive appraisals and questions. Most critical comments regarded interpretation, data collection and methodological design, while compliments were usually about relevance and implications.ConclusionsOnly a small percentage of preprints posted in 2020 in bioRxiv and medRxiv received comments in these platforms. When present, however, these comments address content that is similar to that analyzed by traditional peer review. A more precise taxonomy of peer review functions would be desirable to describe whether post-publication peer review fulfills these roles.