Search papers, labs, and topics across Lattice.
This paper addresses the limitations of the Sketched Isotropic Gaussian Regularizer (SIGReg) in multi-task learning scenarios within the LeWorldModel framework, where it previously led to representation aliasing and poor performance. By applying SIGReg to temporally centered residuals instead of the latent marginal distribution, the authors successfully mitigate the issues of representation collapse and sensitivity to visual perturbations. The proposed method achieves a 1.7x improvement on the LIBERO benchmark for long-horizon tasks and significantly enhances overall success rates across multiple suites, demonstrating a novel approach to stable multi-task world-model learning.
Temporally centered SIGReg boosts multi-task learning performance by 1.7x, revealing critical incompatibilities in latent representation structures.
Recent work on LeWorldModel (LeWM) has shown that the Sketched Isotropic Gaussian Regularizer (SIGReg) enables stable end-to-end world-model learning from pixels by regularizing the latent marginal distribution toward an isotropic Gaussian, thereby preventing representation collapse. While effective and elegant in single-task settings, this recipe does not extend reliably to multi-task training, leading to substantially worse downstream behavior-cloning performance. In this paper, we show that marginal Gaussianization compresses the separation between task-dependent latent clusters relative to within-cluster variation. This compression introduces representation aliasing across tasks and states, and makes the learned representations highly sensitive to small visual perturbations. To address this problem, we apply SIGReg to temporally centered residuals rather than to the latent marginal distribution. This surrogate target places no direct regularization pressure on the separation among cluster centers, removes the requirement that the full latent follow a single isotropic Gaussian, and retains the anti-collapse effect of SIGReg. On the LIBERO benchmark, our method improves downstream success on the long-horizon suite by 1.7x and raises the average success rate across four suites from 53.2% to 73.6%. Without external pretraining, it slightly outperforms Diffusion Policy trained from scratch and approaches the performance of large-scale pretrained policy baselines. These results reveal a structural incompatibility between marginal Gaussian priors and multi-task latent structure, and provide a simple route toward stable and scalable end-to-end multi-task world-model learning.
