Affiliation:
1. School of Electrical and Electronics Engineering, Chung-Ang University, Seoul 06974, Republic of Korea
Abstract
As the field of deep learning experiences a meteoric rise, the urgency to decipher the complex geometric properties of feature spaces, which underlie the effectiveness of diverse learning algorithms and optimization techniques, has become paramount. In this scholarly review, a comprehensive, holistic outlook on the geometry of feature spaces in deep learning models is provided in order to thoroughly probe the interconnections between feature spaces and a multitude of influential factors such as activation functions, normalization methods, and model architectures. The exploration commences with an all-encompassing examination of deep learning models, followed by a rigorous dissection of feature space geometry, delving into manifold structures, curvature, wide neural networks and Gaussian processes, critical points and loss landscapes, singular value spectra, and adversarial robustness, among other notable topics. Moreover, transfer learning and disentangled representations in feature space are illuminated, accentuating the progress and challenges in these areas. In conclusion, the challenges and future research directions in the domain of feature space geometry are outlined, emphasizing the significance of comprehending overparameterized models, unsupervised and semi-supervised learning, interpretable feature space geometry, topological analysis, and multimodal and multi-task learning. Embracing a holistic perspective, this review aspires to serve as an exhaustive guide for researchers and practitioners alike, clarifying the intricacies of the geometry of feature spaces in deep learning models and mapping the trajectory for future advancements in this enigmatic and enthralling domain.
Funder
National Research Foundation of Korea
Subject
General Mathematics,Engineering (miscellaneous),Computer Science (miscellaneous)
Reference115 articles.
1. Deep learning in computer vision: A critical review of emerging techniques and application scenarios;Chai;Mach. Learn. Appl.,2021
2. Zhai, X., Kolesnikov, A., Houlsby, N., and Beyer, L. (2022, January 18–22). Scaling vision transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.
3. Wortsman, M., Ilharco, G., Kim, J.W., Li, M., Kornblith, S., Roelofs, R., Lopes, R.G., Hajishirzi, H., Farhadi, A., and Namkoong, H. (2022, January 18–22). Robust fine-tuning of zero-shot models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.
4. Deep learning in clinical natural language processing: A methodical review;Wu;J. Am. Med. Inform. Assoc.,2020
5. Speech recognition using deep neural networks: A systematic review;Nassif;IEEE Access,2019
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献