Search papers, labs, and topics across Lattice.
This paper introduces ToolRec, a calibrated preference alignment framework designed specifically for query recommendation in on-device intelligent assistants, addressing the limitations of existing methods that focus on standard chatbot interactions. By constructing SysToolKit, a repository of 708 system tools, and implementing a dual-level calibration mechanism to refine click data, ToolRec effectively mitigates noise from user activity variations while enhancing the relevance of tool-invoking queries. Extensive A/B testing on the OPPO Xiaobu platform reveals that ToolRec significantly boosts Click-Through Rate (CTR) and total clicks compared to strong baselines, demonstrating its practical effectiveness in real-world applications.
ToolRec transforms query recommendation for on-device assistants by aligning user preferences with actionable system tools, leading to a significant increase in user engagement.
Large Language Models (LLMs) have significantly advanced generative query recommendation. However, existing alignment methods primarily focus on standard chatbot scenarios, falling short in on-device intelligent assistants where users predominantly expect the rapid invocation of system-level tools. Moreover, directly aligning LLMs with real-world click logs introduces severe noise due to varying user activity levels and the failure to emphasize execution-oriented queries. To address these challenges, we propose ToolRec, a calibrated preference alignment framework tailored for on-device query recommendation. To ground query recommendation with executable actions, we first construct SysToolKit, a comprehensive repository of 708 system tools, paired with a context-aware tool retrieval mechanism to ensure recommendation relevance. We then propose a dual-level calibration mechanism to refine raw click data, effectively mitigating user behavioral noise by calibrating signals based on user activity levels, while simultaneously up-weighting click signals on system-level tool-invoking queries. Guided by these refined preference signals, we then align the model using a sample-level weighted Kahneman-Tversky Optimization (KTO). Extensive online A/B tests on our mobile assistant platform OPPO Xiaobu, which has over 150 million monthly active users, demonstrate that ToolRec can significantly improve Click-Through Rate (CTR) and total clicks volume over strong baselines while maintaining high query relevance.
