Search papers, labs, and topics across Lattice.
The paper introduces PFGNet, a fully convolutional network for spatiotemporal predictive learning that uses frequency-guided peripheral gating to dynamically modulate receptive fields. PFGNet's Peripheral Frequency Gating (PFG) block extracts localized spectral cues and adaptively fuses multi-scale large-kernel peripheral responses with learnable center suppression, effectively creating spatially adaptive band-pass filters. Experiments on Moving MNIST, TaxiBJ, Human3.6M, and KTH datasets demonstrate that PFGNet achieves state-of-the-art or near state-of-the-art forecasting performance with significantly fewer parameters and FLOPs compared to existing methods.
Ditch the RNNs and attention: PFGNet's frequency-guided gating achieves SOTA spatiotemporal prediction with a fully convolutional architecture, slashing parameters and FLOPs.
Spatiotemporal predictive learning (STPL) aims to forecast future frames from past observations and is essential across a wide range of applications. Compared with recurrent or hybrid architectures, pure convolutional models offer superior efficiency and full parallelism, yet their fixed receptive fields limit their ability to adaptively capture spatially varying motion patterns. Inspired by biological center-surround organization and frequency-selective signal processing, we propose PFGNet, a fully convolutional framework that dynamically modulates receptive fields through pixel-wise frequency-guided gating. The core Peripheral Frequency Gating (PFG) block extracts localized spectral cues and adaptively fuses multi-scale large-kernel peripheral responses with learnable center suppression, effectively forming spatially adaptive band-pass filters. To maintain efficiency, all large kernels are decomposed into separable 1D convolutions ($1 \times k$ followed by $k \times 1$), reducing per-channel computational cost from $O(k^2)$ to $O(2k)$. PFGNet enables structure-aware spatiotemporal modeling without recurrence or attention. Experiments on Moving MNIST, TaxiBJ, Human3.6M, and KTH show that PFGNet delivers SOTA or near-SOTA forecasting performance with substantially fewer parameters and FLOPs. Our code is available at https://github.com/fhjdqaq/PFGNet.
