Search papers, labs, and topics across Lattice.
The paper introduces Falcon, a 0.7B parameter vision-language foundation model for remote sensing, designed to handle a variety of tasks through a unified, prompt-based paradigm. To train Falcon, the authors created Falcon_SFT, a large-scale instruction-tuning dataset comprising 78 million samples from 5.6 million remote sensing images with diverse instructions and hierarchical annotations. Experiments across 67 datasets and 14 tasks demonstrate Falcon's strong performance in image classification, object detection, segmentation, and image captioning, despite its relatively small size.
A 0.7B parameter model, Falcon, rivals larger models on remote sensing tasks by using a novel instruction-tuning dataset, Falcon_SFT, proving that data quality can trump model size.
This paper introduces a holistic vision-language foundation model tailored for remote sensing, named Falcon. Falcon offers a unified, prompt-based paradigm that effectively executes comprehensive and complex remote sensing tasks. Falcon demonstrates powerful understanding and reasoning abilities at the image, region, and pixel levels. Specifically, given simple natural language instructions and remote sensing images, Falcon can produce impressive results in text form across 14 distinct tasks, i.e., image classification, object detection, segmentation, image captioning, and etc. To facilitate Falcon's training and empower its representation capacity to encode rich spatial and semantic information, we developed Falcon_SFT, a large-scale, multi-task, instruction-tuning dataset in the field of remote sensing. The Falcon_SFT dataset consists of approximately 78 million high-quality data samples, covering 5.6 million multi-spatial resolution and multi-view remote sensing images with diverse instructions. It features hierarchical annotations and undergoes manual sampling verification to ensure high data quality and reliability. Extensive comparative experiments are conducted, which verify that Falcon achieves remarkable performance over 67 datasets and 14 tasks, despite having only 0.7B parameters. We release the complete dataset, code, and model weights at https://github.com/TianHuiLab/Falcon, hoping to help further develop the open-source community.
