Search papers, labs, and topics across Lattice.
This paper introduces an evidential deep-learning approach to enhance position-based click models by incorporating epistemic uncertainty into the estimation of click probabilities. By outputting beta-distributions for relevance and position-bias variables, the model effectively captures the uncertainty surrounding predictions, addressing a significant limitation of existing frequentist models. Experimental results demonstrate that this approach outperforms standard policy gradients in learning meaningful distributions, thereby improving the understanding of user interactions with rankings.
Capturing epistemic uncertainty in click predictions could revolutionize how we model user interactions with rankings, leading to more robust recommendation systems.
User interactions with rankings are affected by both items' relevances and display positions. Accordingly, click probabilities are often modeled as a product of relevance and position factors; and for improving recommendation and search, one needs to disentangle relevance from position bias. However, existing click models only provide frequentist point-estimates that do not capture any measure of epistemic uncertainty. Consequently, there is no indication of how much confidence one should have in their predictions. In this work, we introduce the first evidential deep-learning approach to form an epistemic alternative to the important position-based click model. Our learned model takes as input item and position features and outputs a beta-distribution for every relevance and position-bias variable of the position-based model. These distributions capture epistemic uncertainty about click probabilities and the underlying effects of attraction and position bias. The main challenge of our approach is its optimization for which we propose approximation and conditioning techniques to provide numerical stability and variance reduction. Our experiments indicate that our approach captures epistemic uncertainty in predictions on previously-unseen data, whereas standard policy gradients fail to learn meaningful distributions. We believe our contribution of the first contextual epistemic click model constitutes an important step in incorporating Bayesian uncertainty into click modeling.
