Time-domain Deep-learning Filtering of Structured Atmospheric Noise for Ground-based Millimeter Astronomy

Abstract

Abstract The complex physics involved in atmospheric turbulence makes it very difficult for ground-based astronomy to build accurate scintillation models and develop efficient methodologies to remove this highly structured noise from valuable astronomical observations. We argue that a deep-learning approach can bring a significant advance to treat this problem because of deep neural networks’ inherent ability to abstract nonlinear patterns over a broad scale range. We propose an architecture composed of long short-term memory cells and an incremental training strategy inspired by transfer and curriculum learning. We develop a scintillation model and employ an empirical method to generate a vast catalog of atmospheric-noise realizations and train the network with representative data. We face two complexity axes: the signal-to-noise ratio (S/N) and the degree of structure in the noise. Hence, we train our recurrent network to recognize simulated astrophysical pointlike sources embedded in three structured-noise levels, with a raw-data S/N ranging from 3 to 0.1. We find that a slow and repetitive increase in complexity is crucial during training to obtain a robust and stable learning rate that can transfer information through different data contexts. We probe our recurrent model with synthetic observational data, designing alongside a calibration methodology for flux measurements. Furthermore, we implement traditional matched filtering (MF) to compare its performance with our neural network, finding that our final trained network can successfully clean structured noise and significantly enhance the S/N compared to raw data and in a more robust way than traditional MF.