Table of Contents

1. Training data augmentation, extending Sec. 3.1 in the paper.
2. Additional quantitative results, extending Tab. 1 in the paper.
3. Qualitative comparison with image-based compositing methods, extending Fig. 4 in the paper.
4. Qualitative comparison with lighting estimation methods, extending Fig. 4 in the paper.
5. Human perceptual study, extending Sec. 5.3 in the paper.
6. Ablation study on different predictors, extending Sec. 5.4 in the paper.
7. Video demo.
8. Failure cases.
9. Additional objects.
10. More real-world 2D compositing.

1. Training data augmentation

As mentionned in Sec. 3.1 of the paper, we keep three intrinsic maps from OpenRooms unchanged (depth, normal and albedo), but randomly remove parts of the shading using one of the following masking strategies.

Note that these examples were seen by the network at training time.

Random masking (60% of the time)

Here, a random combination of rectangles and circles are used to create the shading mask.

Fully masked shading (30% of the time)

Here, no shading information is fed to the network. Therefore, it has to hallucinate the full shading information and relight the whole scene.

Fully known shading (10% of the time)

We also train the network to render an image where all the shading is given.

2. Additional quantitative results

The metrics in Tab. 1 of the paper were computed on the whole image, which is a standard way of quantitatively evaluating compositing results. Since some image-based compositing methods like AnyDoor [12] and ControlCom [85] work in the latent space of Stable Diffusion and don't use a background preserving technique like ours (described in Sec. 3.3), we also evaluate metrics in a way that gives less penalty to the background fidelity. To do so, we compute the same metrics on a rectangular crop defined as the bounding box of the object mask, extended by 25% on each side (while ensuring to avoid going over the image's bounds). This crop therefore includes the full object and most shadows. Note that the LPIPS here is also calculated after resizing the test images and references to 256x256.

The following table shows the metrics computed on the crop for all methods. We observe that our method still beats image-based methods on all metrics. Furthermore, our method is still competitive or better than lighting-based methods. It is important to note that computing metrics on the object crop generally degrades scores for all methods, which is due to the fact that the error is computed on the parts of the image that change the most, i.e. the inserted object and its surroundings.

3. Qualitative comparison with image-based compositing methods

In the following, we show additional results on the evaluation dataset for image-based compositing methods, extending Fig. 4 from the paper. Instead of sorting images by PSNR, as done in the paper, here they are sampled randomly from the test set.

4. Qualitative comparison with lighting estimation methods

In the following, we show additional results on the evaluation dataset from lighting estimation methods, extending Fig. 4 from the paper. Instead of sorting images by PSNR, as done in the paper, here they are sampled randomly from the test set.

5. Human perceptual study

As described in Sec. 5.3, a human perceptual study was conducted to compare 8 techniques (including ours). Here, we show the first page containing the instructions. After clicking "Start", the observer would be shown a page containing all 160 questions. The observer were not told but each pair of image contained one image generated with the ground truth lighting and one image generated using one of the 8 techniques (including ours).

6. Ablation study on different predictors for the background intrinsics

Since we need to estimate the background intrinsics for the image compositing, we conducted an ablation study to compare the performance with different intrinsic predictors. In our method, we choose to use ZoeDepth for depth, StableNormal for normal, and Intrinsic Image Diffusion for albedo.

The following table shows the comparison of different predictors for the background intrinsics. No significant difference is observed between the different predictors, which suggests that ZeroComp is robust to the choice of the background intrinsic predictors.

7. A video of our gradio demo

We provide a video of our gradio demo, which is a web-based interface for users to try our method. The demo allows users to upload their own background and object images, and config the compositing parameters. The demo will then generate the composited image using our method.

8. Failure cases

We identified the two most frequents failure cases of our method: limited specular reflections on non-Lambertian objects and incorrect shadow geometry.

Limited specular reflections on non-Lambertian objects

Our model trained on OpenRooms generates weak reflections on non-Lambertian objects, since the OpenRooms dataset doesn't include objects with diverse glossy or metallic materials. Furthermore, the metallic maps are not available in OpenRooms, leading to ambiguous results. Fortunately, we can improve specular reflections by training ZeroComp on a dataset with more non-Lambertian objects and more intrinsic maps, e.g., Hypersim or InteriorVerse.

Inaccurate shadow geometry

Since ZeroComp receives as input 2D instrinsic maps, the occluded geometry of the object is unknown to the network. As expected, in some cases, the missing geometry information leads to incorrect shadow shapes, as shown in the following examples.

9. Additional objects

Our evaluation set is composed mostly of furniture items inserted into various scenes. Here we show that our model also generates realistic composites with a diverse set of objects.

10. More real-world 2D compositing

Here, we extend Fig. 9 by showing additional examples of real 2D objects composited into real backgrounds. For each example row, we show the 2D object, the target background, and the predicted composite.