Search papers, labs, and topics across Lattice.
This paper presents a unified framework for vision models called RGB In and RGB Out (RINO), which reformulates various forms of visual information鈥攕uch as masks and depth maps鈥攊nto a common RGB image format, allowing for a single model to handle diverse visual tasks. By leveraging a generic image editing architecture without the need for task-specific fine-tuning, RINO achieves competitive zero-shot performance across both dense understanding tasks like segmentation and depth estimation, as well as generation tasks such as pose-to-image conversion. This approach parallels the operational flexibility of language models, suggesting a significant advancement in the development of unified vision-language systems.
Transforming diverse visual tasks into a unified RGB format enables a single model to excel across multiple domains without task-specific tuning.
This work introduces a unified formulation for vision models, where diverse forms of visual information beyond natural images, such as masks, depth maps, and other structured visual signals, are all represented as RGB images, while general visual tasks can be converted into a common RGB-to-RGB image editing problem. In this paradigm, different types of visual information internally share the same encoding and decoding architecture and parameters as natural images, enabling a single model to transfer across tasks through a unified visual interface, in a way analogous to how language models operate over text. We refer to this formulation as RGB In and RGB Out (RINO). Built upon a generic image editing backbone without task-specific fine-tuning, RINO demonstrates robust and competitive zero-shot performance on both dense understanding tasks such as segmentation and depth estimation (where we unify outputs as RGB), and dense-conditioned generation tasks such as pose-to-image generation (where we unify inputs as RGB). We hope this study provides useful insights toward general unified vision-language systems, where diverse visual tasks can be expressed, interpreted, and solved through a shared visual language. Code is available at https://github.com/yangtiming/RINO.
