ATRAcTR (Authentic Transparent Relevant Accurate Track-Record): A Screening Tool to Assess the Potential for Real-World Data Sources to Support Creation of Credible Real-World Evidence for Regulatory Decision-Making

Abstract

Abstract Background: Adoption and use of RWD for decision-making has been complicated by concerns regarding whether RWD was fit-for-purpose or was of sufficient validity to support the creation of credible RWE. This has greater urgency as regulatory agencies begin to use real world evidence (RWE) to inform decisions about treatment effectiveness. Methods: We developed a practical screening tool to assess the quality of RWD sources using the framework of Modern Validity Theory. While there has been some convergence of conceptual frameworks, consensus has yet to emerge regarding how to specifically evaluate whether RWD is reliable and fit-for-purpose. We developed a screening tool consistent with the current frameworks and how researchers generally evaluate existing RWD sources for research that they intend to submit to regulatory agencies. Results: The tool has five dimensions: authenticity, transparency, relevance, accuracy, and reliability. Based upon these dimensions, we developed what more detailed information should be sought by researchers to screen potential RWD sources. Conclusions: Using a hypothetical example of a medical claims data source, we showed that responding to the tool would not require an extraordinary burden or a lengthy document. This RWD screening tool, which is ready for immediate use, is consistent with current conceptual frameworks to assess whether RWD is fit-for-purpose and adds the additional consideration of experience with RWD sources consistent with Modern Validity Theory.