Search papers, labs, and topics across Lattice.
This paper addresses the challenge of skeleton-based action recognition in scenarios with limited field-of-view (FoV), where existing models struggle due to joint visibility dropout. The authors introduce PartialVisGraph, a hypergraph framework that utilizes learnable virtual hyperedges and a visibility prior to enhance feature propagation and maintain robust performance under constrained visibility conditions. Experimental results show that PartialVisGraph achieves state-of-the-art accuracy, with improvements of up to 68.8% in severely restricted FoV scenarios while also outperforming baselines in full-visibility settings.
Action recognition can thrive even in severely limited visibility, with a framework that boosts performance by up to 68.8% in challenging conditions.
Skeleton-based action recognition has achieved remarkable success by exploiting joint coordinates and their topological connections, yet prevailing methods overwhelmingly assume complete and clean skeleton inputs. In real-world deployments, such as egocentric vision, crowded surveillance, wearable devices, or edge robotics, limited field-of-view (FoV) frequently causes substantial joint visibility dropout, leading to severe performance degradation that existing models are largely unprepared to handle. To bridge this critical yet underexplored gap, we introduce PartialVisGraph, a novel hypergraph framework tailored for robust skeleton action recognition under constrained FoV. We first construct highly expressive hypergraphs by introducing learnable virtual hyperedges that form a soft incidence matrix, capturing flexible high-order dependencies beyond conventional pairwise graphs. We then propose the Single-Head Sample-Adaptive Transformer, which adaptively aggregates joint features onto hyperedges while explicitly incorporating a visibility prior. This prior selectively gates information flow, preventing occluded or out-of-view joints from corrupting reliable feature propagation. We further establish rigorous evaluation protocols with realistic FoV simulation benchmarks on NTU RGB+D 60 and 120. Extensive experiments demonstrate that PartialVisGraph consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8\% on subsets with severe FoV restrictions compared to recent strong baselines, while remaining superior on full-visibility settings. Our approach offers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments.
