XGrad: Boosting Gradient-Based Optimizers With Weight Prediction-Reference-Cited by-同舟云学术

XGrad: Boosting Gradient-Based Optimizers With Weight Prediction

Author:

Guan Lei¹^ORCID,Li Dongsheng²^ORCID,Shi Yanqi²^ORCID,Meng Jian¹^ORCID

Affiliation:

1. Department of Mathematics, National University of Defense Technology, Changsha, Hunan, China

2. National Key Laboratory of Parallel and Distributed Computing, National University of Defense Technology, Changsha, Hunan, China

Funder

National Natural Science Foundation of China

State Administration for Science, Technology and Industry for National Defense

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Link

Reference48 articles.

2. On the importance of initialization and momentum in deep learning;Sutskever

3. Adam: A method for stochastic optimization;Kingma;Proc. Int. Conf. Learn. Representations,2015

4. Decoupled weight decay regularization;Loshchilov;Proc. Int. Conf. Learn. Representations,2018

5. Very deep convolutional networks for large-scale image recognition;Simonyan;Proc. Int. Conf. Learn. Representations,2015

Cited by 2 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

2. AdaPlus: Integrating Nesterov Momentum and Precise Stepsize Adjustment on Adamw Basis;ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP);2024-04-14