Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a hierarchical spatiotemporal vector quantization (HSTVQ) framework for unsupervised skeleton-based action segmentation. HSTVQ uses a two-level vector quantization approach, first associating skeletons with sub-actions and then aggregating these into action-level representations. By jointly reconstructing input skeletons and timestamps, HSTVQ achieves state-of-the-art performance on HuGaDB, LARa, and BABEL datasets while also reducing segment length bias.
Unsupervised action segmentation from skeleton data just got a whole lot better: a new method significantly outperforms prior work and reduces bias towards short segments.
We propose a novel hierarchical spatiotemporal vector quantization framework for unsupervised skeleton-based temporal action segmentation. We first introduce a hierarchical approach, which includes two consecutive levels of vector quantization. Specifically, the lower level associates skeletons with fine-grained subactions, while the higher level further aggregates subactions into action-level representations. Our hierarchical approach outperforms the non-hierarchical baseline, while primarily exploiting spatial cues by reconstructing input skeletons. Next, we extend our approach by leveraging both spatial and temporal information, yielding a hierarchical spatiotemporal vector quantization scheme. In particular, our hierarchical spatiotemporal approach performs multi-level clustering, while simultaneously recovering input skeletons and their corresponding timestamps. Lastly, extensive experiments on multiple benchmarks, including HuGaDB, LARa, and BABEL, demonstrate that our approach establishes a new state-of-the-art performance and reduces segment length bias in unsupervised skeleton-based temporal action segmentation.