A backward SDE method for uncertainty quantification in deep learning-Reference-Cited by-同舟云学术

A backward SDE method for uncertainty quantification in deep learning

Published:2022 Issue:10 Volume:15 Page:2807
ISSN:1937-1632
Container-title:Discrete and Continuous Dynamical Systems - S
language:
Short-container-title:DCDS-S

Author:

Archibald Richard¹,Bao Feng²,Cao Yanzhao³,Zhang He⁴

Affiliation:

1. Computational Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

2. Department of Mathematics, Florida State University, Tallahassee, Florida, USA

3. Department of Mathematics and Statistics, Auburn University, Auburn, Alabama, USA

4. School of Mathematics, Jilin University, Changchun, China

Abstract

<p style='text-indent:20px;'>We develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.</p>

Publisher

American Institute of Mathematical Sciences (AIMS)

Subject

Applied Mathematics,Discrete Mathematics and Combinatorics,Analysis

Reference31 articles.

1. R. T. Chen, Y. Rubanova, J. Bettencourt and D. K. Duvenaud, Neural ordinary differential equations, Advances in Neural Information Processing Systems, (2018), 6571–6583.

2. T. Chen, E. B. Fox and C. Guestrin, Stochastic gradient hamiltonian monte carlo, Proceedings of the 31st International Conference on Machine Learning, (2014).

3. B. Dai, A. Shaw, L. Li, L. Xiao, N. He, Z. Liu, J. Chen, L. Song.SBEED: Convergent reinforcement learning with nonlinear function approximation, Proceedings of Machine Learning Research, Stockholmsmässan, Stockholm Sweden, PMLR, 80 (2018), 1125-1134.

4. W. E, J. Han and Q. Li, A mean-field optimal control formulation of deep learning, Research in the Mathematical Sciences, 6 (2019), 41 pp.

5. N. El Karoui, S. Peng, M. C. Quenez.Backward stochastic differential equations in finance, Math. Finance, 7 (1997), 1-71.

Cited by 2 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Numerical Analysis for Convergence of a Sample-Wise Backpropagation Method for Training Stochastic Neural Networks;SIAM Journal on Numerical Analysis;2024-03-01

2. From Optimal Control to Mean Field Optimal Transport via Stochastic Neural Networks;Symmetry;2023-09-08