Search papers, labs, and topics across Lattice.
The paper introduces REFLEX, a train-free evolutionary framework that decouples visual diagnosis from code generation in large multimodal language models (LLMs) to enhance policy search transparency and efficiency. By employing a vision-enabled Critic to generate structured diagnoses and a text-optimized Actor for policy synthesis, REFLEX enables the retention of algorithmic insights across independent runs and provides clear mutation traces. Evaluations on multiple benchmarks show that REFLEX achieves superior sample efficiency, solving complex tasks like Acrobot and Pendulum with fewer LLM calls while maintaining high performance levels.
REFLEX achieves remarkable sample efficiency, solving complex tasks with fewer than 10 LLM calls while ensuring transparent policy evolution.
Large multimodal language models (LLMs) have emerged as powerful tools for guiding evolutionary search toward interpretable programmatic policies. However, existing frameworks rely on a monolithic model call to simultaneously interpret visual behavioral evidence and synthesize corrective code. This diagnosis-repair entanglement creates an opaque feedback loop, obscuring the rationale behind mutations and preventing the retention of algorithmic insights across independent runs. To achieve auditable and efficient policy search, we argue that visual diagnosis must be structurally decoupled from code generation. We present REFLEX, a train-free evolutionary framework that operationalizes this decoupling. In REFLEX, a vision-enabled Critic first distills task-specific behavioral evidence into structured, auditable diagnoses. Subsequently, a text-optimized Actor synthesizes child policies using these diagnoses alongside a persistent, self-evolving Skill Memory of reusable code snippets. This architecture not only provides transparent mutation traces but also enables cross-run programmatic knowledge transfer. Extensive evaluations across control benchmarks (Lunar Lander, Acrobot, Pendulum) and a 36-dimensional antenna array synthesis task demonstrate exceptional sample efficiency. Notably, REFLEX solves Acrobot and Pendulum in under 10 LLM calls and reaches a best Normalized Weighted Score of 1.092 on Lunar Lander, achieving highly competitive final performance while significantly accelerating the early-stage discovery of transparent policies.
