Towards Good Statistical Practice. CORSTAN Validated Questionnaire for Assessing the Correctness of Statistical Analysis in Medical Research

Abstract

BACKGROUND: In evidence-based medicine, the research methodology is determined by the risks of systematic biases and incorrect data analysis. Minimizing both risks increases the internal validity of the study. There are numerous recommendations and guidelines for data analysis and reporting, but the international community has not yet developed a questionnaire for reviewers to assess the quality of statistical analysis.AIM: To develop a tool for formalized assessment of the quality of statistical analysis presented in scientific medical publications.MATERIALS AND METHODS: The questionnaire was developed based on the authors' decades of experience in statistical data analysis and reviewing the statistical aspects of biomedical articles and dissertations. The SAMPL guidelines, ICH E9, and other guidelines were taken into account when developing the questionnaire. Internal validation of the questionnaire was based on an independent assessment by two experts of 20 randomly selected articles on randomized controlled trials (RCTs) from elibrary.ru, and further statistical analysis of the agreement of experts' conclusions.RESULTS: The CORSTAN (CORrect STatistical ANalysis) questionnaire was developed, which consists of two parts: the first part (10 questions) is intended for evaluating studies of any designs, while the second (following eight questions) is for additional assessment of RCTs. A stratification of the risk of incorrect statistical analysis is proposed. The evaluation of the questionnaire's internal validity showed its substantial and almost perfect agreement for each question and each article both in the sum of points and risk level.CONCLUSION: The use of the questionnaire will simplify and harmonize the statistical review of publications and manuscripts in various institutions - scientific journals, dissertation boards, etc. The questionnaire can also be helpful for authors during preparing manuscripts; it will also help improve the quality of publications and research itself. We plan to improve the questionnaire as we gain experience in its application.