Search papers, labs, and topics across Lattice.
SignNet-1M is a large-scale multilingual sign language video dataset designed to enhance the robustness of sign language models against real-world distribution shifts. Utilizing advanced techniques like 3D Gaussian Splatting for novel-view rendering and diffusion models for scene and identity editing, the dataset introduces realistic variations that mimic real-world conditions. Experiments demonstrate that training on SignNet-1M significantly improves model generalization across diverse conditions while preserving in-distribution performance, marking a substantial advancement in sign language recognition and translation tasks.
Training on SignNet-1M boosts sign language model robustness by improving generalization across diverse real-world conditions without sacrificing performance.
Sign language models are typically trained on datasets captured under constrained conditions, with limited viewpoint, background, and signer-identity diversity, leading to poor robustness under real-world distribution shifts. We introduce SignNet-1M, a large-scale augmented dataset spanning ASL, CSL, and German Sign Language (DGS). SignNet-1M synthesizes realistic variations along three axes: (i) novel-view rendering (rotation and zoom) via 3D Gaussian Splatting (3DGS), (ii) scene/identity editing via diffusion models for background replacement and signer substitution while preserving sign motion and linguistic content, and (iii) post-rendering augmentations that emulate capture and compression artifacts (e.g., pose/temporal perturbations and video-level corruptions) to better match in-the-wild recordings. Beyond data release, we provide a unified benchmark suite across downstream tasks (e.g., translation and recognition) and ablations that isolate each augmentation component. Experiments across backbones show that training with SignNet-1M consistently improves generalization under cross-view, cross-background, cross-identity, and post-rendering shifts, while maintaining strong in-distribution performance. The dataset, full augmentation pipeline, and benchmark are available at https://signnet.chatsign.ai/.
Tsinghua AI