Search papers, labs, and topics across Lattice.
This paper introduces FlexLAM, a novel approach that addresses the bottleneck trade-off in Latent Action Models (LAMs) by implementing variable-length latent actions through nested dropout. By allowing for prefix-valid codes, FlexLAM effectively balances the need for compact transition structure with the flexibility to incorporate additional detail when necessary, outperforming traditional fixed-capacity LAMs in both standard and challenging scenarios. The findings demonstrate that FlexLAM not only enhances the latent-action interface but also supports dynamic token-budget adjustments during inference without requiring retraining.
FlexLAM achieves superior performance over fixed-capacity models by dynamically adjusting latent action lengths, revolutionizing how we approach action alignment in video decision-making.
Latent actions provide a compact interface between action-free video and downstream decision-making, yet existing Latent Action Models (LAMs) force every transition through a fixed-capacity bottleneck. We identify a bottleneck trade-off: overly tight codes can discard transition cues needed for action alignment, while overly loose codes preserve additional transition variation that must be resolved when alignment labels are scarce or narrowly distributed. FlexLAM replaces this fixed capacity with variable-length latent actions trained by nested dropout, yielding prefix-valid codes that capture compact transition structure first and add detail only when needed, without new architectures or losses. A single FlexLAM matches or surpasses separately trained fixed-capacity LAMs at every evaluated token budget under standard scarce-label supervision and under a low-return single-task alignment stress test, indicating that FlexLAM is not merely adjustable at inference time but learns a better latent-action interface at the same token budgets. The same model supports inference-time token-budget adjustment without retraining, and FlexLAM improves Ego4D transition reconstruction. These results suggest that variable-length latent actions are an architecture-free, drop-in upgrade to the fixed-capacity bottleneck in latent action models, latent-action world models, and video-pretrained action interfaces.
