Efficient Convex Optimization Requires Superlinear Memory-Reference-Cited by-同舟云学术

Efficient Convex Optimization Requires Superlinear Memory

Published:2024-09-06 Issue: Volume: Page:
ISSN:0004-5411
Container-title:Journal of the ACM
language:en
Short-container-title:J. ACM

Author:

Marsden Annie¹^ORCID,Sharan Vatsal²^ORCID,Sidford Aaron¹^ORCID,Valiant Gregory¹^ORCID

Affiliation:

1. Stanford University, Stanford, United States

2. University of Southern California, Los Angeles, United States

Abstract

We show that any memory-constrained, first-order algorithm which minimizes d -dimensional, 1-Lipschitz convex functions over the unit ball to 1/poly( d ) accuracy using at most d 1.25 − δ bits of memory must make at least

\(\tilde{\Omega }(d^{1 + (4/3)\delta }) \)

first-order queries (for any constant δ ∈ [0, 1/4]). Consequently, the performance of such memory-constrained algorithms are at least a polynomial factor worse than the optimal

\(\tilde{O}(d) \)

query bound for this problem obtained by cutting plane methods that use

\(\tilde{O}(d^2) \)

memory. This resolves one of the open problems in the COLT 2019 open problem publication of Woodworth and Srebro.

Publisher

Association for Computing Machinery (ACM)

Link

https://dl.acm.org/doi/pdf/10.1145/3689208

Reference78 articles.

1. Dan Alistarh, Torsten Hoefler, Mikael Johansson, Nikola Konstantinov, Sarit Khirirat, and Cédric Renggli. 2018. The convergence of sparsified gradient methods. Advances in Neural Information Processing Systems 31 (2018).

2. The Space Complexity of Approximating the Frequency Moments

3. The Volumetric Barrier for Semidefinite Programming

4. Yossi Arjevani and Ohad Shamir. 2015. Communication complexity of distributed convex learning and optimization. Advances in neural information processing systems 28 (2015).