Size-independent sample complexity of neural networks-Reference-Cited by-同舟云学术

Size-independent sample complexity of neural networks

Published:2020-05-04 Issue:2 Volume:9 Page:473-504
ISSN:2049-8772
Container-title:Information and Inference: A Journal of the IMA
language:en
Short-container-title:

Author:

Golowich Noah¹,Rakhlin Alexander²,Shamir Ohad³

Affiliation:

1. Harvard University, Cambridge, MA, USA

2. Department of Brain & Cognitive Science MIT, Cambridge, MA, USA

3. Microsoft Research and Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot, Israel

Abstract

Abstract We study the sample complexity of learning neural networks by providing new bounds on their Rademacher complexity, assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth and, under some additional assumptions, are fully independent of the network size (both depth and width). These results are derived using some novel techniques, which may be of independent interest.

Funder

Harvard University

Publisher

Oxford University Press (OUP)

Subject

Applied Mathematics,Computational Theory and Mathematics,Numerical Analysis,Statistics and Probability,Analysis

Link

http://academic.oup.com/imaiai/article-pdf/9/2/473/33321322/iaz007.pdf

Reference29 articles.

1. Learning and generalization in overparameterized neural networks, going beyond twolayers;Allen-Zhu,2018

2. Universal approximation bounds for superpositions of a sigmoidal function;Barron;IEEE Trans. Inf. Theory,1993

3. Spectrally-normalized margin bounds for neural networks;Bartlett,2017

4. Vapnik–Chervonenkis dimension bounds for two-and three-layer networks;Bartlett;Neural Comput.,1993

Cited by 26 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. On size-independent sample complexity of ReLU networks;Information Processing Letters;2024-08

2. Machine Learning Modeling of Nonlinear Processes with Lyapunov Stability Guarantees;2024 American Control Conference (ACC);2024-07-10

3. Rethinking generalization of classifiers in separable classes scenarios and over-parameterized regimes;2024 International Joint Conference on Neural Networks (IJCNN);2024-06-30

4. Statistical Complexity of Quantum Learning;Advanced Quantum Technologies;2024-04-14

5. Optimal Rates of Approximation by Shallow ReLU$$^k$$ Neural Networks and Applications to Nonparametric Regression;Constructive Approximation;2024-02-26