Structures of neural network effective theories-Reference-Cited by-同舟云学术

Structures of neural network effective theories

Published:2024-05-07 Issue:10 Volume:109 Page:
ISSN:2470-0010
Container-title:Physical Review D
language:en
Short-container-title:Phys. Rev. D

Author:

Banta Ian¹^ORCID,Cai Tianji¹²^ORCID,Craig Nathaniel¹³,Zhang Zhengkang¹³⁴^ORCID

Affiliation:

1. Department of Physics, University of California, Santa Barbara, California 93106, USA

2. SLAC National Accelerator Laboratory, Stanford University, Stanford, California 94309, USA

3. Kavli Institute for Theoretical Physics, University of California, Santa Barbara, California 93106, USA

4. Department of Physics and Astronomy, University of Utah, Salt Lake City, Utah 84112, USA

Abstract

We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations.

Published by the American Physical Society

2024

Funder

U.S. Department of Energy

Aspen Center for Physics

National Science Foundation

Kavli Institute for Theoretical Physics, University of California, Santa Barbara

Publisher

American Physical Society (APS)

Link

http://harvest.aps.org/v2/journals/articles/10.1103/PhysRevD.109.105007/fulltext

Reference22 articles.

1. Bayesian Learning for Neural Networks

2. Wide neural networks of any depth evolve as linear models under gradient descent *