Search papers, labs, and topics across Lattice.
This paper introduces TOWN-VLA, a prompt-authority interface designed to enhance vision-language-action (VLA) manipulation by controlling when and how prompts can alter policy inputs. The authors demonstrate that traditional methods of appending text to prompts can lead to significant performance degradation, a phenomenon termed "prompt-form collapse," where the form of the instruction overshadows its semantic content. By implementing a fixed compatibility rule that allows only authorized prompts to modify the base instruction, TOWN-VLA improves success rates in both simulated and physical environments, achieving a notable increase in task performance across multiple evaluation metrics.
"Prompt-form collapse" can drastically reduce success rates, but TOWN-VLA's controlled intervention boosts performance by ensuring only meaningful prompts are executed.
Retrieval can efficiently and effectively augment a frozen vision--language--action (VLA) policy without retraining, yet retrieved text becomes a control intervention once it enters the executed prompt. In a matched audit, raw appended text reduces mean success from 92.47\% to 3.00\%, while meaningful and length-matched meaningless appends both fail on all 500 states. This result identifies \emph{prompt-form collapse}: changing the instruction form, rather than adding useful semantics, can dominate execution. We introduce TOWN-VLA (Think Only When Needed), a prompt-authority interface that separates candidate generation from permission to alter the policy input. A fixed compatibility rule authorizes a canonical compact instruction; otherwise, the interface restores the original Base prompt exactly. Across 900 audited routes, every route follows this contract: 525 routes recover Base with matching hashes, and all 375 authorized prompts preserve the task signature. On a matched $4\times7$ LIBERO-Plus evaluation with 10{,}030 episodes per method, success rises from 69.5\% to 73.1\% ($+362$ episodes; 95\% CI 1.89--5.45 points), improving on six perturbation axes and all four suites. On a physical PiPER arm with a frozen \pizerofive{} checkpoint, success rises from 52.7\% to 78.7\% over 150 trials per method ($p=3.16\times10^{-6}$). Prompt authority is enforceable for a frozen controller; oracle-free admission calibration is the next deployment target.
Tsinghua AI