EverLight: Indoor-Outdoor Editable HDR Lighting Estimation
Mohammad Reza Karimi Dastjerdi     Jonathan Eisenmann    
Yannick Hold-Geoffroy     Jean-François Lalonde    





[Paper]
[Supplementary]
[Poster]


Accepted in International Conference on Computer Vision (ICCV), 2023!


Abstract

Because of the diversity in lighting environments, existing illumination estimation techniques have been designed explicitly on indoor or outdoor environments. Methods have focused specifically on capturing accurate energy (e.g., through parametric lighting models), which emphasizes shading and strong cast shadows; or producing plausible texture (e.g., with GANs), which prioritizes plausible reflections. Approaches which provide editable lighting capabilities have been proposed, but these tend to be with simplified lighting models, offering limited realism. In this work, we propose to bridge the gap between these recent trends in the literature, and propose a method which combines a parametric light model with 360° panoramas, ready to use as HDRI in rendering engines. We leverage recent advances in GAN-based LDR panorama extrapolation from a regular image, which we extend to HDR using parametric spherical gaussians. To achieve this, we introduce a novel lighting co-modulation method that injects lighting-related features throughout the generator, tightly coupling the original or edited scene illumination within the panorama generation process. In our representation, users can easily edit light direction, intensity, number, etc. to impact shading while providing rich, complex reflections while seamlessly blending with the edits. Furthermore, our method encompasses indoor and outdoor environments, demonstrating state-of-the-art results even when compared to domain-specific methods.


Video - ICCV 2023



Citation

@InProceedings{Dastjerdi_2023_ICCV,
    author    = {Dastjerdi, Mohammad Reza Karimi and Eisenmann, Jonathan and Hold-Geoffroy, Yannick and Lalonde, Jean-Fran\c{c}ois},
    title     = {EverLight: Indoor-Outdoor Editable HDR Lighting Estimation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {7420-7429}
}


Data and results

We provide the full set of results for indoor lighting estimation reported in the paper. Specifically, we compute results for 9 different lighting estimation methods from the recent literature on a test set of 2240 images (extracted from the Laval Indoor HDR Dataset). See the README.md and download the full dataset here (11 GB).


Acknowledgements

The name of this paper EverLight is a homage to the Critical Role’s The Legend of Vox Machina. This work was partially supported by NSERC grant ALLRP557208-20. We thank Sai Bi for his help with extending the dynamic range of the panoramas and everyone at UL who helped with proofreading.