AI-ready data in space science and solar physics: problems, mitigation and action plan

Abstract

In the domain of space science, numerous ground-based and space-borne data of various phenomena have been accumulating rapidly, making analysis and scientific interpretation challenging. However, recent trends in the application of artificial intelligence (AI) have been shown to be promising in the extraction of information or knowledge discovery from these extensive data sets. Coincidentally, preparing these data for use as inputs to the AI algorithms, referred to as AI-readiness, is one of the outstanding challenges in leveraging AI in space science. Preparation of AI-ready data includes, among other aspects: 1) collection (accessing and downloading) of appropriate data representing the various physical parameters associated with the phenomena under study from different repositories; 2) addressing data formats such as conversion from one format to another, data gaps, quality flags and labeling; 3) standardizing metadata and keywords in accordance with NASA archive requirements or other defined standards; 4) processing of raw data such as data normalization, detrending, and data modeling; and 5) documentation of technical aspects such as processing steps, operational assumptions, uncertainties, and instrument profiles. Making all existing data AI-ready within a decade is impractical and data from future missions and investigations exacerbates this. This reveals the urgency to set the standards and start implementing them now. This article presents our perspective on the AI-readiness of space science data and mitigation strategies including definition of AI-readiness for AI applications; prioritization of data sets, storage, and accessibility; and identifying the responsible entity (agencies, private sector, or funded individuals) to undertake the task.