Search papers, labs, and topics across Lattice.
This paper introduces Frequency-Domain Latent-attention Gated Pooling (FLaG), a novel aggregation module that transforms encoder outputs into the Fourier domain for improved token aggregation. By leveraging learnable latent queries and a sample-conditioned channel gate, FLaG enhances representation quality across diverse tasks, including antimicrobial peptide activity prediction, image classification, and language processing. The results show that FLaG outperforms traditional pooling methods, achieving superior performance metrics such as Spearman correlation and accuracy across multiple benchmarks, while revealing unique sensitivity patterns in frequency domains.
FLaG achieves state-of-the-art performance by harnessing the power of frequency-domain representations, revealing how low- and high-frequency sensitivities can be strategically utilized across various tasks.
Token aggregation converts token-level representations into fixed-dimensional sample representations, but most pooling methods operate only in the original token space. We introduce Frequency-Domain Latent-attention Gated Pooling (FLaG), a plug-in aggregation module that re-expresses encoder outputs in the Fourier domain before final pooling. FLaG represents the nonredundant rFFT spectrum through concatenated real and imaginary components, summarizes spectral tokens with learnable latent queries, derives a sample-conditioned channel gate, and reconstructs modulated token representations for downstream aggregation. We evaluate the same architecture across ESM2-based antimicrobial peptide (AMP) activity prediction, ResNet18 image classification on CIFAR-10 and CIFAR-100, and three RoBERTa-based language tasks. FLaG achieves the best macro-averaged Spearman correlation coefficient, RMSE, and Recall@50 across four AMP backbone-species settings and the highest top-1 accuracy on CIFAR 10. It also achieves the best mean results on five of seven language metrics, although mean pooling remains strongest on STSBenchmark. AMP-side mechanistic analyses reveal low-frequency prediction sensitivity across most encoder layers, with increased relative high-frequency sensitivity in the final layer, and pronounced peptide-specific positional responses. The residual gate broadly amplifies spectral channels while preserving the low-frequency-dominated energy profile, whereas latent cross-attention exhibits sample- and species-specific spectral allocation. Overall, FLaG provides a transferable frequency-domain aggregation bias across protein, visual, and textual representations, with benefits that depend on the backbone and downstream task. Supplementary materials, source code, and data are available at https://www.healthinformaticslab.org/supp/ and https://github.com/Kewei2023/AMPCliff/tree/FLaG.
