MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation

2026-09-01

AI Summary

本文针对相机控制视频生成中的尺度依赖问题，提出MeRoPE方法，通过保持特征范数和限制注意力分数来提高生成视频与条件姿态的一致性。

Abstract

In camera-controlled video generation, geometry-aware positional encodings condition tokens on camera extrinsics and per-token viewing rays. Existing schemes, however, have a scale-dependent failure mode on real-world metric camera trajectories: homogeneous projective encodings cause attention logits and feature norms to grow unbounded with physical translation baselines. We propose MeRoPE (Metric Rotary Position Embedding), a norm-preserving relative camera encoding for attention. MeRoPE encodes relative orientations between calibrated viewing rays with orthogonal rotation blocks, maps raw metric displacements into multi-frequency rotary phases, and adds a disparity-anchored correspondence prior along the epipolar arc. This design strictly preserves feature norms, bounds pre-softmax attention logits regardless of the physical translation scale, and maintains exact invariance to global rigid coordinate changes. Across nuScenes and PanShot, which cover large-baseline trajectories and diverse camera optics, respectively, MeRoPE achieves stronger camera control than prior encodings, with the best consistency between generated camera motion and conditioning poses in both rotation and translation. Code will be made publicly available.

Problem

• camera-controlled video generation
• geometry-aware positional encodings
• scale-dependent failure mode
• attention logits
• feature norms

Innovation

• Metric Rotary Position Embedding
• norm-preserving
• relative camera encoding
• attention logits bounded
• invariance to global rigid coordinate changes