Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces IDiom, an autoregressive protein language model trained on a large dataset of predicted intrinsically disordered protein regions (IDRs), and RL-SAE, a novel post-training reinforcement learning method that leverages sparse autoencoder features for controlled sequence generation. IDiom generates diverse IDRs that mimic natural sequence properties, while RL-SAE enables precise control over function-associated sequence patterns by rewarding the activation of specified feature sets. The combined approach significantly outperforms existing methods in activating targeted features and improving predicted biological functions, facilitating interpretable and composable IDR design.
Reinforcement learning with sparse autoencoder features (RL-SAE) unlocks precise, interpretable, and composable design of intrinsically disordered protein regions, activating 90% of targeted functional features compared to 24% with activation steering.
Intrinsically disordered protein regions (IDRs) play central roles in cellular processes such as transcriptional regulation, signal transduction, and subcellular localization, yet their functional design remains challenging. Structure-based design methods do not readily apply to IDRs, and existing protein language models are trained on full-length protein sequences, thus learning a prior that is biased towards folded domains. Here, we present IDiom, an autoregressive protein language model trained on IDiom-DB, a dataset of 54 million predicted IDRs curated from the AlphaFold Database. IDiom generates diverse sequences that recapitulate the composition, patterning, motifs, and predicted disorder of natural IDRs. To control function-associated sequence patterns, we also introduce reinforcement learning with sparse autoencoder features (RL-SAE), a post-training method that rewards the generation of sequences that activate specified feature sets. Across eight IDR design tasks, RL-SAE sequences activate, on average, 90% of 30 targeted features, compared to 24% for activation steering. We demonstrate that RL-SAE improves the predicted subcellular localization and transcriptional activity of generated IDRs compared to steering and supervised fine-tuning, and enables features associated with distinct biological functions to be combined within individual sequences. Thus, IDiom and RL-SAE enable interpretable and composable IDR design through explicit control of function-associated sequence features. More broadly, RL-SAE could extend to other protein design settings where interpretable features provide useful design targets. Code is available at https://github.com/rotskoff-group/idiom.
Stanford HAI