Supplementary material

This supplementary material complements the main paper by providing full videos, technical details and additional results.

Here we show the full video used to make the "backflip" rotation (with no translation) from fig. 2, along with two additional examples of that same motion. On top is overlaid the input camera rotation in dark grey. Notice how only our method achieves a full 360° looping rotation.

Here we show the full videos used to make fig. 1.

Camera motion: Pitch from +80° to -80°, no translation.

Prompt: A quiet mountain village during snowfall as smoke rises from chimneys and lights glow along the snowy street. Many llamas are walking around in the street.

Camera motion: Roll from -90° to 90°, move forward by 8 meters.

Prompt: A tropical beach at sunrise with palm trees swaying gently while small waves roll onto the golden sand.

Here we show training data samples, complementing fig. 4. Each sample displays the video alongside camera trajectory visualizations from three perspectives: top view, side view, and a 3D view. The cameras are color-coded to represent their position in time: the first camera is purple and the last one is red.

Caption: The video begins with a view of a cozy, eclectic shop filled with various vintage and rustic items. A statue of a Native American figure, adorned with a colorful blanket and a beaded necklace, stands prominently on the right. The shop is well-lit, with warm lights casting a welcoming glow on the brick walls and wooden furniture. A Christmas tree decorated with lights is visible in the background, adding a festive touch to the scene. The camera pans slightly to the left, revealing more of the shop's interior, which includes shelves with framed pictures, lamps, and assorted decorative pieces. The statue remains the focal point as the camera moves, capturing the shop's eclectic charm and the cozy atmosphere. Looking down, we see a person holding a phone in their hand.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: People walk under an archway in a historic city square. Looking down, we see a person walking on a cobblestone path.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: The video begins with a view of a modern, spacious living room featuring a sleek glass coffee table, a black floor lamp, and a brown sofa. The room is well-lit with natural light streaming through large windows, and a balcony with a white railing is visible above. In the background, a dining area with red chairs and a round table is partially visible. The camera slowly pans across the room, highlighting the minimalist decor and architectural details, including a large framed picture on the wall. As the camera moves, it reveals more of the room's features, such as a black spherical light fixture on the coffee table and a chessboard set on a stand. The video maintains a static view, focusing on the room's interior design and layout. Looking down, we see a glass bowl with apples on a table.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: People walk on an escalator in a busy indoor setting. Looking down, we see a person with a backpack on an escalator.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: Yellow aircraft with "REINO DE ESPAÑA 43032" flies over a rocky landscape.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: The video begins with a view of a wooden boardwalk alongside a river, where a tall ship with multiple masts is docked. The scene transitions to a bridge with cables, leading to a cityscape with modern buildings under a clear blue sky. The camera moves forward, capturing the river's surface and the bridge's structure. The view shifts to the riverbank, featuring a red brick building with a waterfront restaurant and a crane in the background. A person walks on the boardwalk, and the camera follows them, maintaining the focus on the river and the urban environment. Looking down, we see a person walking on a wooden deck while holding a phone.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: A serene underwater scene unfolds, showcasing a vibrant coral reef bathed in the soft light of the ocean surface. The camera pans gently over the reef, highlighting its diverse textures and colors, with sunlight filtering through the water, creating a tranquil and ethereal atmosphere.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: The video captures a vibrant outdoor scene at a flower garden on a sunny day. A wide path runs through the garden, bordered by rows of red and purple flowers. People are scattered throughout the area, some walking along the path, while others gather in groups to admire the flowers. In the background, a cityscape with tall buildings is visible under a clear blue sky. A kite is flying in the sky, adding to the leisurely atmosphere. The camera moves along the path, providing a panoramic view of the garden and its visitors. Looking down, we see a person walking on a path surrounded by red and purple flowers.

Video

Null Pitch Video

Top View

Side View

3D View

Caption: Paragliding over a vast desert landscape with mountains in the distance.

Video

Null Pitch Video

Top View

Side View

3D View

Obtaining absolute camera poses

For completeness, here we repeat equation 2, which computes the absolute camera extrinsics: $$ \begin{gather} \mathbf{t}_{\text{abs},f} = \psi(\mathbf{\tilde{u}}_0)\mathbf{t}_{\text{rel},f} \nonumber \\ \mathbf{R}_{\text{abs},f} = \mathbf{R}_{\text{pano},f} \psi(\mathbf{\tilde{u}}_0)\mathbf{R}_{\text{rel},f} \\ \mathbf{E}_{\text{abs},f} = \varphi(\mathbf{R}_{\text{pano},0}) [\mathbf{R}_{\text{abs},f}|\mathbf{t}_{\text{abs},f} ] \;, \nonumber \end{gather} $$ where $\mathbf{\tilde{u}}_0$ corresponds to the average up vector of the first frame, $\mathbf{R}_{\text{pano},f}$ corresponds to the sampled camera rotation for frame $f$ and $\mathbf{t}_{\text{rel},f}$ and $\mathbf{R}_{\text{rel},f}$ are the translation and rotational components of the relative camera poses respectively, derived from SfM. We now define exactly the function $\psi(\cdot)$, which aligns camera poses to the gravity, and $\varphi(\cdot)$, which rotates the poses so that the yaw is null for the first frame.

Definition of the gravity alignment function $\psi(\cdot)$

Definition of the yaw removal function $\varphi(\cdot)$

We begin by extracting from $\mathbf{R}_{\text{pano},0}$ the rotations along the $y$, $x$ and $z$ axis, for the yaw, pitch and roll matrices. This results in the following decomposition: $$ \mathbf{R}_{\text{pano},0} = \mathbf{R}_{\text{pano},0,y} \mathbf{R}_{\text{pano},0,x} \mathbf{R}_{\text{pano},0,z}. $$ We then simply use the inverse yaw rotation $$ \varphi(\mathbf{R}_{\text{pano},0}) \gets \mathbf{R}_{\text{pano},0,y}^{-1}. $$

Training dataset (extends sec. 3.2)

Below we show the distributions of rotations and translations, comparing our training set to RealEstate10K, a dataset typically used for training camera control methods (e.g. AC3D, GEN3C), and PanShot (used to train UCPE). The translation rose plots show the distribution of translation directions seen from the top (obtained from the last frame).

Rotation diversity

Translation diversity (top view)

Evaluation dataset (extends sec. 4.3)

Here, we supply additional details and statistics on our evaluation benchmark, SpatialVID-extreme, extending sec. 4.3 of the paper.

Camera path randomization

We randomly sample a start roll from \([-40°, 40°]\), and a start pitch from \([-90°, 90°]\). The final roll and pitch are sampled the same way, and the end yaw is sampled from \([-180°, 180°]\). The intermediate rotations are interpolated using spherical linear interpolation. Since the method used to compute absolute orientation metrics (Perspective Fields) is never trained with rolls going beyond 45° in magnitude, we resample a random rotation trajectory if an intermediate frame goes outside the roll bounds of \([-40°, 40°]\). Finally, we obtain the translations by estimating them from the original videos with ViPE. We then apply a global rotation to those translations around the yaw axis, with an angle sampled from \([0°, 360°]\), in order to reduce the bias towards translations moving forward.

Evaluation dataset statistics

Here we show that the original SpatialVID-HQ dataset provides limited diversity in absolute orientation and relative rotations. Our new evaluation benchmark, SpatialVID-extreme, provides a broader coverage of Euler angles and total angular distance.

Evaluation dataset statistics

Training details

We start by showing an example sampled from the training data. In the training video (left), we can see part of the selfie stick used to hold the camera, but it is absent from the null pitch video (center), used for captioning. As explained in the paper, if undesirable elements are present in the training video (such as the selfie stick), but not in the caption, the model will learn to generate these as if they were normal scene content. To prevent this, we generate a third set of videos, this time looking straight down (right), and also caption them.

In practice, we give the following instructions to the VLM to obtain the "look down" prompt:

We then simply concatenate the two prompts 50% of the time in the following way:

if random.random() < 0.5:
    full_caption = caption_null_pitch + " " + caption_down
else:
    full_caption = caption_null_pitch
            

With the "Look down" caption included, the full caption is:

Inference details

At inference time, we keep the input (positive) prompt intact. In order to ensure that undesirable elements (e.g. hands, distortions) are absent from the generated video, we use the following prompt for the negative direction:

Qualitative comparison

Here we show a few examples from our evaluation where we clearly see the artefacts removed by training on the "look down" prompts. The same prompt and negative prompt are used at inference. In the following three examples, we see that, without integrating the "look down" prompts in the training process, the cameraman appears when the camera is pointed directly down.

Quantitative comparison

We compare our full model (trained for 30k iterations) against an ablated version trained without look-down prompts (trained for 28k iterations). Despite the slightly shorter training schedule of the ablated model, the improvements from integrating look-down prompts are clearly visible in the higher CLIP score, confirming the effectiveness of this training strategy.

Our lightweight camera control encoder module is composed of the following modules:

Panorama generation methods, such as PanoWAN [43] can generate realistic 360° equirectangular videos. As mentioned in the paper, gravity-aligned camera control can be achieved by cropping the desired field of view out of the generated panorama, but this comes with several disadvantages: 1) most of the panorama pixels are discarded, resulting in severe resolution degradation; 2) the desired concepts described in the prompt can be cropped out and thus not appear in the video; and 3) current text-conditioned methods do not provide control over camera translation.

Qualitative comparison

Here, we show an example where two concepts should be present in the generated image: a house and a lake.

We run both our method and PanoWAN on 4 different seeds and display the first generated frame. When cropping a 90° FOV into the generated panorama, notice that for 3 out of the 4 seeds (0, 1, 2), PanoWAN only includes either the lake or the house but not both. When cropping a 45° FOV, all seeds lack either the house or the lake. Our method includes both elements, while also generating sharper details. Please zoom in to compare.

	Ours			PanoWAN
	90° FOV	45° FOV		Generated panorama	90° FOV (cropped)	45° FOV (cropped)
Seed 0
Seed 1
Seed 2
Seed 3

We show a second example where a temple should be clearly visible and surrounded by valleys. Again, PanoWAN omits the temple in most generations, whereas our method consistently includes it.

	Ours			PanoWAN
	90° FOV	45° FOV		Generated panorama	90° FOV (cropped)	45° FOV (cropped)
Seed 0
Seed 1
Seed 2
Seed 3

Quantitative comparison

Since PanoWAN produces full 360° panoramas at $896 \times 448$ resolution, we must crop its outputs to create standard perspective images. This cropping results in significant detail loss, discarding 86.1% and 95.8% of pixels for 90° and 45° fields of view, respectively. The reduction in high-frequency details affects CLIP alignment and FID scores, even though both models are trained on similar data. We train ours on a subset of PanoVid, while PanoWAN uses the entire dataset. At 90° FOV, we observe a decrease in prompt alignment (CLIP score of 20.63 vs. 19.46) and image quality (FID score of 115.58 vs. 123.10). When using a narrower FOV of 45°, these differences are more pronounced, with CLIP scores of 18.54 versus 16.82 and FID scores of 122.43 versus 142.81 for PanoWAN.

For the AC3D baselines, we provide the model with the absolute camera orientation through text, as mentioned in sec. 4.4 of the paper. Here, we show the code used to generate these camera descriptions. We first take the absolute camera extrinsics \(E_\text{abs}\) for the video and convert them to Euler angles. We then describe textually only the first frame's pitch and roll and the last frame's yaw, pitch, and roll. Please click below for the full implementation.

def get_camera_prompt_from_absolute_c2w(c2w_absolute):
    euler_angles = c2w_to_pitch_roll_yaw(c2w_absolute)
    first_pitch = euler_angles['pitch'][0].item()
    first_roll = euler_angles['roll'][0].item()
    first_yaw = euler_angles['yaw'][0].item()
    last_pitch = euler_angles['pitch'][-1].item()
    last_roll = euler_angles['roll'][-1].item()
    last_yaw = euler_angles['yaw'][-1].item()
    
    
    def describe_angle_shot(pitch):
        """Describe pitch angle shot."""
        pitch_rounded = round(pitch)
        if -5 <= pitch_rounded <= 5:
            return "near straight-on shot"
        elif pitch_rounded > 0:
            if 5 < pitch_rounded <= 20:
                return f"small tilt-up of {pitch_rounded} degrees"
            elif 20 < pitch_rounded <= 45:
                return f"large tilt-up of {pitch_rounded} degrees"
            else:
                return f"extreme tilt-up of {pitch_rounded} degrees"
        else:
            abs_pitch = abs(pitch_rounded)
            if 5 < abs_pitch <= 20:
                return f"small tilt-down of {abs_pitch} degrees"
            elif 20 < abs_pitch <= 45:
                return f"large tilt-down of {abs_pitch} degrees"
            else:
                return f"extreme tilt-down of {abs_pitch} degrees"
    
    def describe_dutch_angle(roll):
        """Describe roll (Dutch angle) with clockwise/counterclockwise."""
        roll_rounded = round(roll)
        if -5 <= roll_rounded <= 5:
            return "near level shot"
        
        abs_roll = abs(roll_rounded)
        if abs_roll <= 20:
            magnitude = "small"
        elif abs_roll <= 45:
            magnitude = "large"
        else:
            magnitude = "extreme"
        
        # Positive roll is counterclockwise, negative is clockwise
        direction = "counterclockwise" if roll_rounded > 0 else "clockwise"
        
        return f"a {magnitude} Dutch angle tilted {direction} {abs_roll} degrees"
    
    def describe_yaw(yaw):
        """Describe yaw (pan) direction."""
        yaw_rounded = round(yaw)
        
        abs_yaw = abs(yaw_rounded)
        direction = "right" if yaw_rounded > 0 else "left"
        return f"pan of {abs_yaw} degrees turned {direction}"
    
    # Build start description
    start_parts = []
    start_pitch_desc = describe_angle_shot(first_pitch)
    start_roll_desc = describe_dutch_angle(first_roll)
    
    if start_pitch_desc != "near straight-on shot":
        start_parts.append(start_pitch_desc)
    if start_roll_desc != "near level shot":
        start_parts.append(start_roll_desc)
    
    # Build end description
    end_parts = []
    end_yaw_desc = describe_yaw(last_yaw)
    end_pitch_desc = describe_angle_shot(last_pitch)
    end_roll_desc = describe_dutch_angle(last_roll)
    
    if end_yaw_desc:
        end_parts.append(end_yaw_desc)
    if end_pitch_desc != "near straight-on shot":
        end_parts.append(end_pitch_desc)
    if end_roll_desc != "near level shot":
        end_parts.append(end_roll_desc)
    
    # Construct final description
    description_parts = []
    

    start_text = "The camera starts at " if len(start_parts) == 1 else "The camera starts with "
    description_parts.append(start_text + ", and ".join(start_parts))
    
    end_text = "The camera ends with " + ", ".join(end_parts)
    description_parts.append(end_text)
    
    return ". ".join(description_parts)[:-1]
        

After obtaining the camera description, we concatenate it after the regular prompt. Refer to the qualitative results at the end of the supplementary material for examples of camera descriptions.

Additional quantitative results (extends fig. 6.b)

First, we report additional graphs showcasing the relationship between the prompt alignment and the input pitch angle, via the CLIP score, where a higher score indicates a high correlation between the prompt and the image.

We report the CLIP score computed between the generated image and the forward-looking input prompt (left) and the caption from the ground truth at the input pitch angle (right). We further display both the relative metrics shifted to each method's average CLIP score (top) and absolute (bottom) graphs.

We observe that our method (in orange) comes closer to the ground truth values (in black) than by training without null-pitch conditioning (in red) and other methods, across the range of pitch angles.

CLIP computed between the caption from the ground truth (at the input pitch angle) and the generated image (relative variations)

CLIP computed between the input caption (forward looking) and the generated image (relative variations)

CLIP computed between the caption from the ground truth (at the input pitch angle) and the generated image (absolute)

CLIP computed between the input caption (forward looking) and the generated image (absolute)

Additional qualitative results (extends fig. 6.a)

Here, we randomly sampled 6 scenes out of the 20 scenes used in our benchmark. We show the prompt and each method's output along with the ground truth, for 5 different pitch angles. Notice how our method more closely follows the input pitch angle, especially at extreme pitch angles (-90° and 90°).

Note that PreciseCam was trained on PolyHaven panoramas used in our benchmark.

Prompt: Two people are working at a desk with computers. The room has wooden beams on the ceiling and posters on the wall. One person is typing on a keyboard while the other looks at the screen. The workspace is cluttered with various items.

Pitch	Ground truth	Ours	w/o null pitch	UCPE	PreciseCam
90°
50°
0°
-50°
-90°

Prompt: The scene shows an abandoned concrete structure with graffiti on the walls and ceiling. Sunlight streams through large openings, illuminating the overgrown vegetation outside. The interior is dusty and empty, with visible cracks and decay. The graffiti includes the word "RAVE" in red.

Pitch	Ground truth	Ours	w/o null pitch	UCPE	PreciseCam
90°
50°
0°
-50°
-90°

Prompt: A dirt path winds through a lush, green forest with tall trees and dense shrubs. The sky is clear and blue, with a few scattered clouds. The path leads into the distance, surrounded by vibrant vegetation.

Pitch	Ground truth	Ours	w/o null pitch	UCPE	PreciseCam
90°
50°
0°
-50°
-90°

Prompt: A cozy room features a beige tufted sofa in the center, flanked by a Christmas tree and a white fireplace adorned with a wreath. To the right, a gray canopy tent and a small cloud-shaped decoration are visible. The room is illuminated by a chandelier and overhead lights, with sheer curtains on the left and decorative screens nearby.

Pitch	Ground truth	Ours	w/o null pitch	UCPE	PreciseCam
90°
50°
0°
-50°
-90°

Prompt: A large circular softbox light is positioned on the right side of a plain, white room. The softbox emits a bright, diffused light, illuminating the area around it. The ceiling and walls are smooth and unadorned, with visible wiring and a corner of the room slightly curved. The scene remains static, focusing solely on the lighting setup.

Pitch	Ground truth	Ours	w/o null pitch	UCPE	PreciseCam
90°
50°
0°
-50°
-90°

Prompt: A spacious workshop with large windows and industrial equipment is shown. A blue metal structure dominates the foreground, surrounded by various tools and workbenches. In the background, a staircase leads to an upper level with a glass enclosure. The room is filled with scattered tools, machinery, and workstations, creating an organized yet busy atmosphere.

Pitch	Ground truth	Ours	w/o null pitch	UCPE	PreciseCam
90°
50°
0°
-50°
-90°

Here are additional qualitative results, complementing fig. 5. We show 18 samples from our SpatialVID-extreme dataset. The black overlay represents the input camera orientation. For "AC3D + cam. text" baselines, we further provide the camera prompt to the model to indicate the absolute camera orientation to the model.

To avoid overly large file sizes, each video's resolution is reduced from $720 \times 480$ to $480\times 320$.