Search papers, labs, and topics across Lattice.
This paper introduces SimpleDesign, a novel multi-modal protein design model that operates directly in the data space, eliminating the need for the traditional multi-stage training process. By employing a Mixture-of-Transformer architecture, SimpleDesign effectively integrates sequence and structure modalities through a unified end-to-end objective that combines discrete cross-entropy and regression. The model demonstrates superior performance on co-design and unconditional generation tasks, achieving strong results on a dataset of over 2 million sequence-structure pairs, thereby advancing the field of protein engineering and drug discovery.
SimpleDesign achieves state-of-the-art protein design without the cumbersome multi-stage training, streamlining the process and enhancing performance.
Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Developing generative models capable of understanding this intrinsically multi-modal relationship is crucial for fields like drug discovery and protein engineering. Existing models often rely on a multi-stage training process where autoencoders that tokenize data into latent representations are trained in a first stage. Secondly, a generative model is trained on the latent representation of the autoencoder(s), i.e., generative modeling in a latent space. We hypothesize that this multi-stage training is not necessary to obtain performant co-design models and thus present SimpleDesign, an effective multi-modal protein design model trained directly in the data space. SimpleDesign leverages a single-stage end-to-end objective that combines discrete cross-entropy for sequences and a regression objective for structures. In order to effectively model the difference in sequence and structure modalities, we develop a Mixture-of-Transformer architecture that allows modality-specific processing while keeping global self-attention over both modalities. We train SimpleDesign on over 2M sequence-structure pairs achieving strong performance across co-design and unconditional sequence/structure generation benchmarks.
