Search papers, labs, and topics across Lattice.
This paper introduces ConfGuide, a Learning to Guide (LtG) approach for hybrid decision making that generates succinct, targeted textual guidance using conformal risk control to select a subset of relevant outcomes. By focusing guidance on a limited set of outcomes with a bounded false negative rate, ConfGuide aims to improve human decision quality and reduce cognitive load compared to methods that consider all possible outcomes. Experiments on a multi-label medical diagnosis task demonstrate the potential of ConfGuide for generating more digestible and effective guidance.
Conformal risk control lets AI assistants provide targeted advice without overwhelming humans, leading to better decisions.
Building on recent advances in AI, hybrid decision making (HDM) holds the promise of improving human decision quality and reducing cognitive load. We work in the context of learning to guide (LtG), a recently proposed HDM framework in which the human is always responsible for the final decision: rather than suggesting decisions, in LtG the AI supplies (textual) guidance useful for facilitating decision making. One limiting factor of existing approaches is that their guidance compounds information about all possible outcomes, and as a result it can be difficult to digest. We address this issue by introducing ConfGuide, a novel LtG approach that generates more succinct and targeted guidance. To this end, it employs conformal risk control to select a set of outcomes, ensuring a cap on the false negative rate. We demonstrate our approach on a real-world multi-label medical diagnosis task. Our empirical evaluation highlights the promise of ConfGuide.
