Deep Sky Modeling for Single Image Outdoor Lighting Estimation

We propose a data-driven learned sky model, which we use for outdoor lighting estimation from a single image. As no large-scale dataset of images and their corresponding ground truth illumination is readily available, we use complementary datasets to train our approach, combining the vast diversity of illumination conditions of SUN360 with the radiometrically calibrated and physically accurate Laval HDR sky database. Our key contribution is to provide a holistic view of both lighting modeling and estimation, solving both problems end-to-end. From a test image, our method can directly estimate an HDR environment map of the lighting without relying on analytical lighting models. We demonstrate the versatility and expressivity of our learned sky model and show that it can be used to recover plausible illumination, leading to visually pleasant virtual object insertions. To further evaluate our method, we capture a dataset of HDR 360° panoramas and show through extensive validation that we significantly outperform previous state-of-the-art.


Paper

Yannick Hold-Geoffroy, Akshaya Athawale, and Jean-Fran├žois Lalonde
Deep Sky Modeling for Single Image Outdoor Lighting Estimation
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[arXiv pre-print] [BibTeX coming soon]


Supplementary material

We provide additional results in this supplementary page.

Data

The Outdoor HDR ULaval dataset used in this publication is available at http://outdoor.hdrdb.com/.

Poster

Poster and slides coming soon!

Acknowledgements

The authors gratefully acknowledge the following funding sources:

  • A generous donation from Adobe to Jean-Francois Lalonde
  • NVIDIA Corporation with the donation of the Titan X GPU used for this research.
  • NSERC Discovery GRANT RGPIN-2014-05314
  • REPARTI Strategic Network

Ulaval logo
Adobe logo