Search papers, labs, and topics across Lattice.
This paper introduces MORE, an adaptive Multi-Objective Reinforcement learning framework designed for e-commerce dialogue systems that simultaneously optimizes reasoning accuracy and linguistic naturalness. By treating reasoning functions as constraints rather than mixing rewards, MORE effectively mitigates oscillations and instability during learning, leading to more coherent and contextually accurate responses. Experimental results demonstrate that MORE significantly enhances conversion rates and user satisfaction in real-world applications, outperforming strong baselines and achieving substantial improvements in online settings.
Achieving a 30% boost in conversion rates, MORE redefines how dialogue systems balance reasoning and naturalness without compromising performance.
Dialogue systems in e-commerce scenarios often need to satisfy multiple objectives: accurately reasoning over user profiles (e.g., eligibility, credit limit) to ensure correct decision-making and user state interpretation, while also generating natural and faithful responses. These goals are complementary but not identical. In this work, we propose MORE, an adaptive Multi-Objective REinforcement learning framework that jointly optimizes reasoning accuracy and linguistic naturalness. Our preliminary experiments show that directly mixing rewards with diverging optimization dynamics can cause oscillations and unstable learning. Thus, instead of optimizing a single mixed reward, we treat reasoning functions as constraints that guide policy optimization. At inference time, the system directly generates responses without explicit reasoning steps, while still benefiting from reasoning-enhanced scaffold and avoiding additional inference overhead. To better balance linguistic objectives during response generation, we introduce an adaptive multi-reward mechanism that aggregates signals such as fluency and naturalness and dynamically reweighs them via gradient feedback. We evaluate MORE on two real-world dialogue systems at ByteDance and the MultiWOZ 2.2 benchmark, where it consistently outperforms strong baselines. In 14-day online experiments on ByteDance production traffic, MORE improves overall and reached conversion by 16.53% and 30.09%, while increasing user satisfaction and reducing handoff rates. Notably, in a human-machine comparison, MORE recovers about 60% of the incremental conversion lift achieved by human agents.
