Search papers, labs, and topics across Lattice.
The paper introduces OPERA, a reinforcement learning-based agent for image restoration that jointly optimizes tool selection (planning) and tool application (execution) in an end-to-end manner. OPERA addresses limitations of prior agent-based methods by directly optimizing tool composition using RL and co-training restoration tools to encourage cooperative behavior. Experiments show OPERA outperforms both all-in-one models and existing agent-based methods on multi-degradation benchmarks and real-world datasets.
End-to-end joint optimization of planning and execution in an image restoration agent unlocks significantly improved performance compared to independently trained tools and all-in-one models.
Real-world image restoration is challenging due to complex and interacting mixed degradations. Recent agent-based approaches address this problem by composing multiple task-specific restoration tools. However, empirical analysis reveals that their performance is fundamentally limited by implicitly constrained planning spaces and the lack of coordination among independently pretrained tools. To address these issues, we propose OPERA (Optimized Planning-Execution Restoration Agent), a framework that jointly optimizes restoration planning and tool execution in an end-to-end manner. On the planning side, OPERA uses reinforcement learning to directly optimize tool composition over a combinatorial plan space, with the final restoration quality as the reward. On the execution side, OPERA introduces agent-guided co-training of restoration tools, enabling them to learn cooperative behaviors under sequential composition. Extensive experiments on multi-degradation benchmarks and real-world datasets demonstrate that OPERA consistently outperforms both all-in-one restoration models and existing agent-based methods across diverse and complex degradation scenarios.
