ACM Transactions on Graphics · 2022 · 13 citations · 59 references
ArchitectureBuilding Science3D ModelingComputer VisionRenderingArchitectural LightingImage SynthesisLighting IntegrationLighting ModelsEnvironmental Lighting EffectsInpaintingMaterial ReplacementScene RepresentationSemantic MapsSemantic Scene UnderstandingSpecular ReflectionSpecular Reflection Models
We describe a novel approach to decompose a single panorama of an empty indoor environment into four appearance components: specular, direct sunlight, diffuse and diffuse ambient without direct sunlight. Our system is weakly supervised by automatically generated semantic maps (with floor, wall, ceiling, lamp, window and door labels) that have shown success on perspective views and are trained for panoramas using transfer learning without any further annotations. A GAN-based approach supervised by coarse information obtained from the semantic map extracts specular reflection and direct sunlight regions on the floor and walls. These lighting effects are removed via a similar GAN-based approach and a semantic-aware inpainting step. The appearance decomposition enables multiple applications including sun direction estimation, virtual furniture insertion, floor material replacement, and sun direction change, providing an effective tool for virtual home staging. We demonstrate the effectiveness of our approach on a large and recently released dataset of panoramas of empty homes.
59
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, Trevor Darrell · 2015 · 36.9K citations
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2017 · 22.1K citations
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
1993 · 14.3K citations
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Proceedings of the 21st International Conference on Neural Information Processing Systems
Daphne Koller, Dale Schuurmans, Yoshua Bengio et al. · 2008 · 4.5K citations