Search papers, labs, and topics across Lattice.
This paper investigates the application of TextGrad, a method that utilizes natural-language feedback to optimize text components in language models, specifically in the context of agent-based systems. The authors find a significant gap between the ability to follow a useful policy and the ability to learn that policy from experience, highlighting the challenges faced when feedback is only available after a sequence of actions. Notably, while human-written policies enhance the performance of two frozen 7B agents in TextWorldExpress by 5.0 success points, policies derived from agent trajectories fail to consistently outperform fixed prompting methods.
Human-written policies can boost agent performance significantly, but learning from experience remains a major hurdle for effective text policy generation.
TextGrad improves language-model systems by revising text from feedback. Its core thesis is that natural-language feedback can act as a gradient for optimizing text components without changing model weights. Applying it to agents is harder because feedback arrives only after a sequence of actions, making it difficult to identify which decision caused failure. We study this problem by separating the ability to follow a useful policy from the ability to learn that policy from experience. Our main finding is a clear gap between these two abilities. Human-written policies improve two frozen 7B agents on TextWorldExpress by 5.0 success points, showing that useful policy text exists. However, policies generated from agent trajectories do not reliably outperform fixed prompting, even with richer traces, counterfactual evidence, or iterative GEPA search. The main challenge for agent-level TextGrad is therefore not executing textual policy updates, but reliably generating and selecting them from experience.
NVIDIA