Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent-Reference-Cited by-同舟云学术

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Published:2023-10-05 Issue: Volume: Page:
ISSN:2096-6385
Container-title:Communications on Applied Mathematics and Computation
language:en
Short-container-title:Commun. Appl. Math. Comput.

Author:

Whiting Wes^ORCID,Wang Bao,Xin Jack

Abstract

AbstractWe prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Funder

Directorate for Mathematical and Physical Sciences

U.S. Department of Energy

Publisher

Springer Science and Business Media LLC

Subject

Computational Mathematics,Applied Mathematics

Link

https://link.springer.com/content/pdf/10.1007/s42967-023-00302-9.pdf

Reference12 articles.

1. Bécigneul, G., Ganea, O.-E.: Riemannian adaptive optimization methods. arXiv:1810.00760 (2019)

2. Bonnabel, S.: Stochastic gradient descent on Riemannian manifolds. IEEE Trans. Autom. Control 58(9), 2217–2229 (2013)

3. De Sa, C., Gu, A., Ré, C., Sala, F.: Representation tradeoffs for hyperbolic embeddings. CoRR, arXiv:1804.03329 (2018)

4. Ganea, O.-E., Bécigneul, G., Hofmann, T.: Hyperbolic neural networks. CoRR, arXiv:1805.09112 (2018)

5. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980v9 (2014)

Cited by 1 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Heterogeneous Network Embedding Method Based on Hyperbolic Space;2024 5th International Conference on Information Science, Parallel and Distributed Systems (ISPDS);2024-05-31