|
Recent work has shown that diffusion models can be used as powerful neural rendering engines that can be
leveraged for inserting virtual objects into images. Unlike typical physics-based renderers, however,
neural rendering engines are limited by the lack of manual control over the lighting setup, which is
often essential for improving or personalizing the desired image outcome. In this paper, we show that
precise lighting control can be achieved for object relighting simply by specifying the desired shadows
of the object. Rather surprisingly, we show that injecting only the shadow of the object into a
pre-trained diffusion-based neural renderer enables it to accurately shade the object according to the
desired light position, while properly harmonizing the object (and its shadow) within the target
background image. Our method, SpotLight, leverages existing neural rendering approaches and achieves
controllable relighting results with no additional training. Specifically, we demonstrate its use with
two neural renderers from the recent literature. We show that SpotLight achieves superior object
compositing results, both quantitatively and perceptually, as confirmed by a user study, outperforming
existing diffusion-based models specifically designed for relighting.
|
