Altmetrics can capture research evidence: an analysis across types of studies in COVID-19 literature

Abstract

COVID-19 has greatly impacted science. It has become a global research front that constitutes a unique phenomenon of interest for the scientometric community. Accordingly, there has been a proliferation of descriptive studies on COVID-19 papers using altmetrics. Social media metrics serve to elucidate how research is shared and discussed, and one of the key points is to determine which factors are well-conditioned altmetric values. The main objective of this study is to analyze whether the altmetric mentions of COVID-19 medical studies are associated with the type of study and its level of evidence. Data were collected from the PubMed and Altmetric.com databases. A total of 16,672 publications by study types (e.g., case reports, clinical trials, or meta-analyses) that were published in the year 2021 and that had at least one altmetric mention were retrieved. The altmetric indicators considered were Altmetric Attention Score (AAS), news mentions, Twitter mentions, and Mendeley readers. Once the dataset of COVID-19 had been created, the first step was to carry out a descriptive study. Then, a normality hypothesis was evaluated by means of the Kolmogorov–Smirnov test, and since this was significant in all cases, the overall comparison of groups was performed using the nonparametric Kruskal–Wallis test. When this test rejected the null hypothesis, pairwise comparisons were performed with the Mann–Whitney U test, and the intensity of the possible association was measured using Cramer’s V coefficient. The results suggest that the data do not fit a normal distribution. The Mann–Whitney U test revealed coincidences in five groups of study types: The altmetric indicator with most coincidences was news mentions, and the study types with the most coincidences were the systematic reviews together with the meta-analyses, which coincided with four altmetric indicators. Likewise, between the study types and the altmetric indicators, a weak but significant association was observed through the chi-square and Cramer’s V. It can thus be concluded that the positive association between altmetrics and study types in medicine could reflect the level of the “pyramid” of scientific evidence.