UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme
-
Published:2023-09-21
Issue:18
Volume:15
Page:4634
-
ISSN:2072-4292
-
Container-title:Remote Sensing
-
language:en
-
Short-container-title:Remote Sensing
Author:
Yan Yiming12ORCID, Zhou Weikun12, Su Nan12ORCID, Zhang Chi12ORCID
Affiliation:
1. College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China 2. Key Laboratory of Advanced Marine Communication and Information Technology, Ministry of Industry and Information Technology, Harbin Engineering University, Harbin 150001, China
Abstract
While recent advances in the field of neural rendering have shown impressive 3D reconstruction performance, it is still a challenge to accurately capture the appearance and geometry of a scene by using neural rendering, especially for remote sensing scenes. This is because both rendering methods, i.e., surface rendering and volume rendering, have their own limitations. Furthermore, when neural rendering is applied to remote sensing scenes, the view sparsity and content complexity that characterize these scenes will severely hinder its performance. In this work, we aim to address these challenges and to make neural rendering techniques available for 3D reconstruction in remote sensing environments. To achieve this, we propose a novel 3D surface reconstruction method called UniRender. UniRender offers three improvements in locating an accurate 3D surface by using neural rendering: (1) unifying surface and volume rendering by employing their strengths while discarding their weaknesses, which enables accurate 3D surface position localization in a coarse-to-fine manner; (2) incorporating photometric consistency constraints during rendering, and utilizing the points reconstructed by structure from motion (SFM) or multi-view stereo (MVS), to constrain reconstructed surfaces, which significantly improves the accuracy of 3D reconstruction; (3) improving the sampling strategy by locating sampling points in the foreground regions where the surface needs to be reconstructed, thus obtaining better detail in the reconstruction results. Extensive experiments demonstrate that UniRender can reconstruct high-quality 3D surfaces in various remote sensing scenes.
Funder
National Natural Science Foundation of China Heilongjiang Outstanding Youth Foundation
Subject
General Earth and Planetary Sciences
Reference42 articles.
1. Griwodz, C., Gasparini, S., Calvet, L., Gurdjos, P., Castan, F., Maujean, B., Lillo, G.D., and Lanthony, Y. (October, January 28). AliceVision Meshroom: An open-source 3D reconstruction pipeline. Proceedings of the 12th ACM Multimedia Systems Conference, Istanbul, Turkey. 2. MicMac—A free, open-source solution for photogrammetry;Rupnik;Open Geospat. Data Softw. Stand.,2017 3. Labatut, P., Pons, J.P., and Keriven, R. (2007, January 14–21). Efficient Multi-View Reconstruction of Large-Scale Scenes using Interest Points, Delaunay Triangulation and Graph Cuts. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil. 4. Accurate, Dense, and Robust Multiview Stereopsis;Furukawa;IEEE Trans. Pattern Anal. Mach. Intell.,2010 5. Schönberger, J.L., and Frahm, J.M. (2016, January 27–30). Structure-from-Motion Revisited. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.
|
|