Distributed sparse normal means estimation with sublinear communication

Abstract

Abstract We consider the problem of sparse normal means estimation in a distributed setting with communication constraints. We assume there are $M$ machines, each holding $d$-dimensional observations of a $K$-sparse vector $\boldsymbol \mu $ corrupted by additive Gaussian noise. The $M$ machines are connected in a star topology to a fusion center, whose goal is to estimate the vector $\boldsymbol \mu $ with a low communication budget. Previous works have shown that to achieve the centralized minimax rate for the $\ell _2$ risk, the total communication must be high—at least linear in the dimension $d$. This phenomenon occurs, however, at very weak signals. We show that at signal-to-noise ratios (SNRs) that are sufficiently high—but not enough for recovery by any individual machine—the support of $\boldsymbol \mu $ can be correctly recovered with significantly less communication. Specifically, we present two algorithms for distributed estimation of a sparse mean vector corrupted by either Gaussian or sub-Gaussian noise. We then prove that above certain SNR thresholds, with high probability, these algorithms recover the correct support with total communication that is sublinear in the dimension $d$. Furthermore, the communication decreases exponentially as a function of signal strength. If in addition $KM\ll \tfrac{d}{\log d}$, then with an additional round of sublinear communication, our algorithms achieve the centralized rate for the $\ell _2$ risk. Finally, we present simulations that illustrate the performance of our algorithms in different parameter regimes.