Search papers, labs, and topics across Lattice.
This paper introduces an adaptive inference framework for Vision-Language-Action (VLA) models that dynamically routes execution based on task complexity. The approach uses the VLA's vision-language backbone to detect task complexity by projecting latent embeddings into parametric and non-parametric estimators, enabling the model to either act immediately, reason, or abstain from execution. Experiments on LIBERO, LIBERO-PRO, and a real robot demonstrate that a vision-only configuration achieves high F1-score with minimal training data, proving its efficacy as a task complexity detector.
VLA models can achieve 80% F1-score in task complexity detection using only visual embeddings and 5% of training data, suggesting language may be surprisingly redundant for this specific capability.
Current research on Vision-Language-Action (VLA) models predominantly focuses on enhancing generalization through established reasoning techniques. While effective, these improvements invariably increase computational complexity and inference latency. Furthermore, these mechanisms are typically applied indiscriminately, resulting in the inefficient allocation of resources for trivial tasks while simultaneously failing to provide the uncertainty estimation necessary to prevent catastrophic failure on out-of-distribution tasks. Inspired by human cognition, we propose an adaptive framework that dynamically routes VLA execution based on the complexity of the perceived state. Our approach transforms the VLA's vision-language backbone into an active detection tool by projecting latent embeddings into an ensemble of parametric and non-parametric estimators. This allows the system to execute known tasks immediately (Act), reason about ambiguous scenarios (Think), and preemptively halt execution when encountering significant physical or semantic anomalies (Abstain). In our empirical analysis, we observe a phenomenon where visual embeddings alone are superior for inferring task complexity due to the semantic invariance of language. Evaluated on the LIBERO and LIBERO-PRO benchmarks as well as on a real robot, our vision-only configuration achieves 80% F1-Score using as little as 5% of training data, establishing itself as a reliable and efficient task complexity detector.
