Search papers, labs, and topics across Lattice.
This paper introduces PaGE, a novel gaze estimation model that effectively captures the intricate interactions between scene and head features to enhance gaze target prediction. By leveraging a large ViT-H+ backbone as a teacher model and distilling lighter student models from a diverse unlabeled dataset, PaGE achieves state-of-the-art performance, surpassing human benchmarks in 7 out of 9 evaluation metrics. The approach not only reduces the human-AI performance gap by at least 60% in the remaining metrics but also ensures that the distilled models are lightweight enough for practical applications in robotics and consumer devices.
Surpassing human performance in gaze estimation, PaGE closes the human-AI gap by over 60% while remaining lightweight for real-world applications.
Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial reasoning using human appearance (e.g. pose, eye orientation). As a result, human-level performance remains elusive for existing models, limiting their practical application. To this end, we propose PaGE (Practical Gaze Estimator), a gaze estimation model that explicitly models the complex interaction between scene and head features. Using a PaGE model with a large ViT-H+ backbone as the teacher, we further distill student models with lighter backbones on a much larger and more diverse unlabeled dataset. The architectural improvements and novel training recipe allow PaGE to achieve state-of-the-art performance on several gaze estimation tasks, outperforming humans in 7 out of 9 metrics while reducing the human-AI gap by at least 60% in the remaining 2. The distilled student models retain most of the teacher's performance while being lightweight enough for practical deployment on robots and consumer devices. The code and model checkpoints are available at our project page.
Tsinghua AI