Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
Ego-Forge removes the inference-time text prompt and heavy geometric-attention biases previously required to synthesize egocentric video from third-person footage. Eliminating these hand-crafted geometric priors and hallucinated text descriptions circumvents reconstruction error propagation while preserving full visual context. Trained on an order-of-magnitude more multi-view data with a lightweight depth prior and internal dynamic conditioning, the framework achieves state-of-the-art fidelity and faster inference on Ego-Exo4D.
Exocentric-to-egocentric video synthesis no longer requires explicit 3D geometric attention or human-written prompts; scaled diffusion can implicitly resolve complex viewpoint shifts using only internal representations and a lightweight depth prior.
Exo-to-egocentric video generation aims to synthesize what a person sees from their own viewpoint given third-person footage and a target head trajectory. The task requires transferring appearance and semantics across large viewpoint changes while hallucinating content never observed by the exocentric camera. Existing approaches either impose additional input requirements, such as a ground-truth initial egocentric frame or multiple synchronized exocentric views, or remain limited to category-specific settings. EgoX is the first to address cross-activity and in-the-wild generalization, but requires a human-provided caption of the non-existent egocentric view at inference and introduces a computationally expensive geometry-guided attention bias that can propagate reconstruction errors and suppress textual and visual context. We therefore propose \textbf{Ego-Forge}, a caption-free and bias-free framework for exo-to-egocentric generation. It introduces \textit{Dynamic Captioning}, which derives conditioning tokens directly from the model's hidden states and adapts them to the diffusion timestep and network depth, replacing external text conditioning. By scaling training by an order of magnitude and using all available exocentric viewpoints, Ego-Forge learns cross-view correspondence implicitly and eliminates the need for geometry-guided attention, requiring only a lightweight depth prior. Ego-Forge achieves state-of-the-art performance on Ego-Exo4D, runs faster end-to-end, requires no external annotation at inference, and generalizes to in-the-wild scenes, including cases where over-reliance on geometry blocks appearance inference. Our model and source code will be made publicly available.