Search papers, labs, and topics across Lattice.
CinemaTraj introduces a novel framework for generating camera trajectories in 3D scenes by leveraging large language models (LLMs) and structured scene graphs. This approach addresses the limitations of existing methods by integrating spatial reasoning with cinematographic semantics, allowing for the decomposition of user prompts into atomic camera movements. Evaluations on real-world environments demonstrate that CinemaTraj achieves superior prompt alignment, trajectory quality, and safety metrics compared to traditional techniques.
By framing camera trajectory generation as a language-grounded spatial reasoning problem, CinemaTraj achieves high-quality, collision-free cinematographic outputs that align closely with user prompts.
Automatically generating cinematically expressive camera trajectories through 3D scenes from natural language descriptions is a challenging task of high practical value, with applications ranging from real-estate advertising to virtual tour creation. Existing methods either lack true 3D spatial awareness by relying on 2D image priors, or treat trajectory generation as a geometric path planning problem divorced from cinematographic semantics. We present CinemaTraj, a framework that reframes camera trajectory planning as a language-grounded spatial reasoning problem. Given a set of RGB-D images and a user prompt, CinemaTraj equips an LLM agent with a structured 3D scene graph: the agent decomposes the prompt into a sequence of atomic cinematographic movements (dolly, orbit, crane, pan, tilt, zoom, arc). Each movement is instantiated via a novel parametric trajectory representation that is both cinematographically expressive and optimizable for collision avoidance. The scene graph acts as a structured spatial prior, grounding the agent's reasoning in accurate geometric and semantic knowledge of the environment. CinemaTraj further generates synchronized voiceover and subtitles aligned with camera motion, producing narrated cinematic video outputs. We evaluate CinemaTraj on real-world ScanNet++ environments, and show that it produces prompt-faithful, collision-free trajectories with high cinematographic quality, outperforming existing approaches on prompt alignment, trajectory quality, and safety metrics.
