Search papers, labs, and topics across Lattice.
This paper introduces CanonSLR, a novel framework for Continuous Sign Language Recognition (CSLR) that enhances robustness against viewpoint variations by employing a frontal-view-anchored teacher-student learning strategy. The method utilizes Sequence-Level Soft-Target Distillation to effectively transfer temporal knowledge from frontal views to non-frontal samples, addressing challenges like gloss boundary ambiguity and category confusion. Extensive experiments on newly created multi-view benchmarks show that CanonSLR significantly outperforms existing methods, particularly in handling challenging non-frontal views.
CanonSLR achieves unprecedented robustness in sign language recognition by effectively bridging the gap between frontal and non-frontal viewpoints.
Continuous Sign Language Recognition (CSLR) has achieved remarkable progress in recent years; however, most existing methods are developed under single-view settings and thus remain insufficiently robust to viewpoint variations in real-world scenarios. To address this limitation, we propose CanonSLR, a canonical-view guided framework for multi-view CSLR. Specifically, we introduce a frontal-view-anchored teacher-student learning strategy, in which a teacher network trained on frontal-view data provides canonical temporal supervision for a student network trained on all viewpoints. To further reduce cross-view semantic discrepancy, we propose Sequence-Level Soft-Target Distillation, which transfers structured temporal knowledge from the frontal view to non-frontal samples, thereby alleviating gloss boundary ambiguity and category confusion caused by occlusion and projection variation. In addition, we introduce Temporal Motion Relational Enhancement to explicitly model motion-aware temporal relations in high-level visual features, strengthening stable dynamic representations while suppressing viewpoint-sensitive appearance disturbances. To support multi-view CSLR research, we further develop a universal multi-view sign language data construction pipeline that transforms original single-view RGB videos into semantically consistent, temporally coherent, and viewpoint-controllable multi-view sign language videos. Based on this pipeline, we extend PHOENIX-2014T and CSL-Daily into two seven-view benchmarks, namely PT14-MV and CSL-MV, providing a new experimental foundation for multi-view CSLR. Extensive experiments on PT14-MV and CSL-MV demonstrate that CanonSLR consistently outperforms existing approaches under multi-view settings and exhibits stronger robustness, especially on challenging non-frontal views.
