Multi-View Stereo Network Based on Attention Mechanism and Neural Volume Rendering
-
Published:2023-11-10
Issue:22
Volume:12
Page:4603
-
ISSN:2079-9292
-
Container-title:Electronics
-
language:en
-
Short-container-title:Electronics
Author:
Zhu Daixian1ORCID, Kong Haoran1, Qiu Qiang1, Ruan Xiaoman1, Liu Shulin2
Affiliation:
1. College of Communication and Information Engineering, Xi’an University of Science and Technology, Xi’an 710054, China 2. College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
Abstract
Due to the presence of regions with weak textures or non-Lambertian surfaces, feature matching in learning-based Multi-View Stereo (MVS) algorithms often leads to incorrect matches, resulting in the construction of the flawed cost volume and incomplete scene reconstruction. In response to this limitation, this paper introduces the MVS network based on attention mechanism and neural volume rendering. Firstly, we employ a multi-scale feature extraction module based on dilated convolution and attention mechanism. This module enables the network to accurately model inter-pixel dependencies, focusing on crucial information for robust feature matching. Secondly, to mitigate the impact of the flawed cost volume, we establish a neural volume rendering network based on multi-view semantic features and neural encoding volume. By introducing the rendering reference view loss, we infer 3D geometric scenes, enabling the network to learn scene geometry information beyond the cost volume representation. Additionally, we apply the depth consistency loss to maintain geometric consistency across networks. The experimental results indicate that on the DTU dataset, compared to the CasMVSNet method, the completeness of reconstructions improved by 23.1%, and the Overall increased by 7.3%. On the intermediate subset of the Tanks and Temples dataset, the average F-score for reconstructions is 58.00, which outperforms other networks, demonstrating superior reconstruction performance and strong generalization capability.
Funder
National Natural Science Foundation of China Shaanxi Provincial Key R&D General Industrial Project Xi’an Beilin District Science and Technology Plan Project
Subject
Electrical and Electronic Engineering,Computer Networks and Communications,Hardware and Architecture,Signal Processing,Control and Systems Engineering
Reference38 articles.
1. Campbell, N.D., Vogiatzis, G., Hernández, C., and Cipolla, R. (2008, January 12–18). Using multiple hypotheses to improve depth-maps for multi-view stereo. Proceedings of the European Conference on Computer (ECCV), Marseille, France. 2. Accurate, Dense, and Robust Multiview Stereopsis;Ponce;IEEE Trans. Pattern Anal. Mach. Intell.,2010 3. Galliani, S., Lasinger, K., and Schindler, K. (2015, January 7–13). Massively parallel multiview stereopsis by surface normal diffusion. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile. 4. Schönberger, J.L., Zheng, E., Frahm, J.-M., and Pollefeys, M. (2016, January 11–14). Pixelwise view selection for unstructured multi-view stereo. Proceedings of the European Conference on Computer (ECCV), Amsterdam, The Netherlands. 5. Xu, Q., and Tao, W. (2019, January 16–20). Multi-scale geometric consistency guided multi-view stereo. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.
|
|