Author:
Szekeres Béla J.,Izsák Ferenc
Abstract
AbstractImplicit neural networks and the related deep equilibrium models are investigated. To train these networks, the gradient of the corresponding loss function should be computed. Bypassing the implicit function theorem, we develop an explicit representation of this quantity, which leads to an easily accessible computational algorithm. The theoretical findings are also supported by numerical simulations.
Publisher
Springer Science and Business Media LLC
Reference22 articles.
1. El Ghaoui L, Gu F, Travacca B, Askari A, Tsai A (2021) Implicit deep learning. SIAM J Math Data Sci 3(3):930–958. https://doi.org/10.1137/20M1358517
2. Bianchini M, Gori M, Sarti L, Scarselli F (2006) Recursive neural networks and graphs: dealing with cycles. In: Apolloni B, Marinaro M, Nicosia G, Tagliaferri R (eds) Neural nets. Springer, Berlin, Heidelberg, pp 38–43
3. Bianchini M, Gori M, Scarselli F (2002) Recursive processing of cyclic graphs. In: Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN’02 (Cat. No.02CH37290), vol 1, pp 154–1591. https://doi.org/10.1109/IJCNN.2002.1005461
4. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605
5. Kawaguchi K (2021) On the theory of implicit deep learning: global convergence with implicit layers. In: International Conference on Learning Representations. https://openreview.net/forum?id=p-NZIuwqhI4