Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an agentic framework utilizing large vision-language models (LVLMs) to enhance the process of hyperspectral unmixing by refining outputs from conventional modular pipelines. By iteratively gathering spectral and spatial evidence and adjusting the endmember set through merge and discard operations, the framework improves the accuracy of endmember cardinality and abundance estimation. Experimental results demonstrate that this approach consistently outperforms traditional methods while remaining competitive with integrated end-to-end unmixing techniques across various datasets.
Tool-using LVLM agents can significantly enhance hyperspectral unmixing by intelligently refining outputs from existing modular pipelines, leading to more accurate material decomposition.
Hyperspectral unmixing is a key task in remote sensing that aims to decompose mixed pixels in hyperspectral images into their constituent material signatures, or endmembers, and their fractional abundances. Conventional modular approaches estimate the scene composition through successive model-order estimation, endmember extraction, and abundance estimation stages, whose errors can lead to redundant or ambiguous candidate components and ultimately affect the recovered decomposition. We introduce an algorithm-agnostic, large vision-language model (LVLM)-driven agentic framework that refines the outputs of such pipelines rather than replacing their underlying numerical algorithms. Starting from an initial decomposition, the agent iteratively gathers complementary spectral and spatial evidence through dedicated tools, including spectral-library retrieval and abundance-map visualization, and modifies the active endmember set through merge and discard operations followed by abundance re-estimation. We apply the same refinement procedure to several modular pipelines combining different model-order, extraction, and abundance-estimation methods, and evaluate it on HYDICE Urban, Jasper Ridge, and Stonewall Playa. Experiments show that the proposed agent consistently improves endmember cardinality and generally improves the recovered spectral signatures and abundance maps across heterogeneous modular pipelines, while remaining competitive with integrated end-to-end unmixing methods, including CNN-AE, uDAS, and R-CoNMF. These results highlight the potential of tool-using LVLM agents to combine spectral and spatial evidence for algorithm-agnostic refinement of physically grounded hyperspectral unmixing decompositions. Code is publicly available at https://anonymous.4open.science/r/agentic-hu.