Going beyond journal classification for evaluation of research outputs

Abstract

PurposeSeeks to characterise world astronomy research during the last decade by an analysis of papers in the Science Citation Index identified with a special filter and to study Indian output in order to identify the leading institutions and authors.Design/methodology/approachLists of specialist journals and title words of papers were selected to create a filter giving high precision and recall for astronomy papers. Some biology papers were erroneously retrieved because of ambiguous title words. Potential citation impact was determined from journal citation scores, and multiple regression was used to evaluate leading countries.FindingsTitle words added almost a quarter to the list of papers in specialist journals, and the final file contained over 96,000 papers. Potential impact increased with more authors per paper and more addresses; it was greater for papers from Canada, the UK and the USA, and less for papers from China, India and Russia; for other countries the effects of the author's location on potential impact were not statistically significant. Indian astronomy output has increased in potential impact, partly through greater international co‐authorship, but also through indigenous papers.Research limitations/implicationsThe study was confined to one subject area, and impact was determined on the basis of journals, not of individual papers.Practical implicationsUse of title words in addition to journal lists is essential to sub‐field definition in order to have high precision and recall. Because of the confounding effects of authorship numbers, it is necessary to use multiple regression analysis in order to see whether research from a given country is significantly better or worse than average.Originality/valueCharacterises world astronomy research during the last decade by an analysis of papers in the Science Citation Index identified with a special filter.