A lifted Bregman formulation for the inversion of deep neural networks

Author:

Wang Xiaoyu,Benning Martin

Abstract

We propose a novel framework for the regularized inversion of deep neural networks. The framework is based on the authors' recent work on training feed-forward neural networks without the differentiation of activation functions. The framework lifts the parameter space into a higher dimensional space by introducing auxiliary variables, and penalizes these variables with tailored Bregman distances. We propose a family of variational regularizations based on these Bregman distances, present theoretical results and support their practical application with numerical examples. In particular, we present the first convergence result (to the best of our knowledge) for the regularized inversion of a single-layer perceptron that only assumes that the solution of the inverse problem is in the range of the regularization operator, and that shows that the regularized inverse provably converges to the true inverse if measurement errors converge to zero.

Funder

Alan Turing Institute

Publisher

Frontiers Media SA

Subject

Applied Mathematics,Statistics and Probability

Reference58 articles.

1. Deep learning;LeCun;Nature,2015

2. Deep inside convolutional networks: Visualising image classification models and saliency maps;Simonyan;Proceedings of the International Conference on Learning Representations,2014

3. Interpretable explanations of black boxes by meaningful perturbation;Fong;Proceedings of the IEEE International Conference on Computer Vision,2017

4. Explaining image classifiers by counterfactual generation ChangCH CreagerE GoldenbergA DuvenaudD arXiv [Preprint]2019

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3