Search papers, labs, and topics across Lattice.
RIBOSPAN is a 1.61-billion-parameter bidirectional RNA foundation model that effectively addresses the limitations of existing RNA models by enabling complete-transcript modeling with context lengths of up to 10,240 nucleotides. The model employs dense bidirectional self-attention and single-nucleotide tokenization, achieving state-of-the-art performance in nucleotide reconstruction and RNA representation quality across various RNA types. Notably, RIBOSPAN maintains strong contextual responsiveness and representation separation, even under heavy corruption, while also facilitating advanced mRNA generation and redesign through a novel discrete-diffusion framework.
RIBOSPAN achieves unprecedented RNA modeling capabilities by maintaining high-resolution context across full-length transcripts, outperforming existing models in both reconstruction and representation quality.
Full-length RNAs, particularly messenger RNAs, often exceed the context lengths used to pretrain existing RNA foundation models, limiting complete-transcript modeling at single-nucleotide resolution. We present RIBOSPAN, a 1.61-billion-parameter bidirectional RNA foundation model natively pretrained with context lengths up to 10,240 nt. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs. We evaluate the model through nucleotide reconstruction, a controlled long-context representation benchmark, and frozen RNA-type representation analysis. Native 10K pretraining preserves strong reconstruction at 10,240 tokens, while continued pretraining with 40% masking improves recovery under heavy corruption while preserving representation quality. The long-context benchmark further shows that native 10K models maintain strong contextual responsiveness and context-specific representation separation while keeping perturbation-induced representation changes highly localized. Inference-time YaRN scaling recovers much of the contextual organization lost by direct extrapolation of short-context models, but induces substantially greater distal representation diffusion. Frozen-representation evaluations further demonstrate state-of-the-art RNA representation quality, with RIBOSPAN achieving the strongest overall performance across diverse RNA types and retaining a clear advantage on long RNAs. Building on the same backbone, we develop a multidimensionally conditioned discrete-diffusion framework for full-length mRNA generation and redesign, including synonymous-codon diffusion for protein-preserving CDS optimization. Together, RIBOSPAN establishes a powerful long-context foundation for transferable RNA representation learning and full-transcript mRNA design.
