Search papers, labs, and topics across Lattice.
This paper introduces CtrlBench-Rec, a novel collaborative multi-agent framework designed to evaluate the controllability of recommender systems, addressing the critical gap in current evaluation paradigms. By formalizing three key tasks鈥攖arget content discovery, interest profile shaping, and popularity bias mitigation鈥攖he authors systematically assess how well these systems can respond to explicit user commands and mitigate biases. Experiments on real-world datasets reveal significant limitations in the systems' ability to steer recommendations towards long-tail content, highlighting the need for improved controllability in algorithmic design.
Recommender systems resist steering towards long-tail content, revealing a major flaw in their controllability that could impact user experience and algorithmic fairness.
Recommender systems operate as Black-Boxes, leaving users and regulators unable to steer their outputs toward specific intentions or audit their behavior. This lack of controllability, defined as the system's ability to respond to explicit guidance, remains an unaddressed dimension in existing evaluation paradigms. To fill this gap, we propose CtrlBench-Rec, a collaborative multi-agent framework for systematic assessment of controllability. We formalize three fundamental tasks: target content discovery, interest profile shaping, and popularity bias mitigation, which together measure steerability from explicit commands to implicit representation steering and finally to overcoming algorithmic biases.Extensive experiments on real-world datasets and multiple recommendation models demonstrate that our framework effectively quantifies controllability and exposes critical system bottlenecks, most notably persistent resistance to guiding long tail content. CtrlBench-Rec provides the first standardized toolkit for controllable recommendation research, algorithmic auditing, and user empowerment. Our code is released on https://github.com/caskcsg/CtrlBenchRec.
