Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces FaStR, a novel approach to reinforcement learning that utilizes a three-mode tensor representation of the transition kernel, allowing for a low-rank CP decomposition. By employing a noise contrastive objective, FaStR learns separate encoders for states, actions, and next states, resulting in a more compact hypothesis class and significantly reduced sample requirements. Empirical results show that FaStR excels in high-dimensional locomotion tasks, demonstrating effective transferability of the learned state encoder across different actuator configurations.
FaStR reduces sample complexity in reinforcement learning by leveraging a three-mode tensor representation, achieving remarkable performance gains in high-dimensional locomotion tasks.
Learning a compact model of the world from interaction data is central to sample-efficient deep reinforcement learning. Spectral representation methods have become the leading paradigm for representation learning in continuous control by taking a matrix view of the transition kernel, with state-action pairs on one side and next states on the other, and learning a low-rank factorization through self-supervised contrastive objectives. We take this view one step further. The transition kernel is naturally a three-mode tensor over states, actions, and next states, and a CP decomposition gives one feature map per mode. We propose FaStR, which fits this decomposition with a noise contrastive objective, producing separate state, action, and next-state encoders that together form a single spectral representation. The factored form yields a smaller hypothesis class, and the sample size needed for representation learning shrinks by a factor that scales with the smaller of the state and action dimensions. Empirically, FaStR delivers its largest gains on high-dimensional locomotion tasks whose dynamics align with the factored structure, and the learned state encoder transfers intact across actuator shift while only the action encoder is retrained.