Affiliation:
1. Department of Electrical and Computer Engineering Seoul National University
2. Department of Computer Science Dartmouth College
Abstract
AbstractWe propose IBL‐NeRF, which decomposes the neural radiance fields (NeRF) of large‐scale indoor scenes into intrinsic components. Recent approaches further decompose the baked radiance of the implicit volume into intrinsic components such that one can partially approximate the rendering equation. However, they are limited to representing isolated objects with a shared environment lighting, and suffer from computational burden to aggregate rays with Monte Carlo integration. In contrast, our prefiltered radiance field extends the original NeRF formulation to capture the spatial variation of lighting within the scene volume, in addition to surface properties. Specifically, the scenes of diverse materials are decomposed into intrinsic components for rendering, namely, albedo, roughness, surface normal, irradiance, and prefiltered radiance. All of the components are inferred as neural images from MLP, which can model large‐scale general scenes. Especially the prefiltered radiance effectively models the volumetric light field, and captures spatial variation beyond a single environment light. The prefiltering aggregates rays in a set of predefined neighborhood sizes such that we can replace the costly Monte Carlo integration of global illumination with a simple query from a neural image. By adopting NeRF, our approach inherits superior visual quality and multi‐view consistency for synthesized images as well as the intrinsic components. We demonstrate the performance on scenes with complex object layouts and light configurations, which could not be processed in any of the previous works.
Funder
Korea Institute for Advancement of Technology
Ministry of Trade, Industry and Energy
Subject
Computer Graphics and Computer-Aided Design
Reference39 articles.
1. BossM. BraunR. JampaniV. BarronJ. T. LiuC. LenschH.: Nerd: Neural reflectance decomposition from image collections. InProceedings of the IEEE/CVF International Conference on Computer Vision(2021) pp.12684–12694. 2
2. Intrinsic images in the wild
3. BitterliB.:Rendering resources 2016.https://benedikt-bitterli.me/resources/. 5
4. BossM. JampaniV. BraunR. LiuC. BarronJ. T. LenschH. P.: Neural-pil: Neural pre-integrated lighting for reflectance decomposition. InAdvances in Neural Information Processing Systems (NeurIPS)(2021). 2
5. BiS. XuZ. SrinivasanP. MildenhallB. SunkavalliK. HašanM. Hold-GeoffroyY. KriegmanD. RamamoorthiR.: Neural reflectance fields for appearance acquisition.arXiv preprint arXiv:2008.03824(2020). 2 4 6