Search papers, labs, and topics across Lattice.
The paper introduces a per-video unsupervised action segmentation method that approximates the underlying frame distribution using deep kernel learning. It leverages Maximum Mean Discrepancy (MMD) in a Neural Tangent Kernel (NTK) space to define a geometry-preserving metric between the original video distribution and its approximation. The approach achieves competitive results on six standard benchmarks, outperforming prior agglomerative methods in scenarios with unknown segment numbers.
Forget optimal transport – MMD with Neural Tangent Kernels offers a faster, easier-to-optimize path to unsupervised video action segmentation with competitive accuracy.
This work focuses on per-video unsupervised action segmentation, which is of interest to applications where storing large datasets is either not possible, or nor permitted. We propose to segment videos by learning in deep kernel space, to approximate the underlying frame distribution, as closely as possible. To define this closeness metric between the original video distribution and its approximation, we rely on maximum mean discrepancy (MMD) which is a geometry-preserving metric in distribution space, and thus gives more reliable estimates. Moreover, unlike the commonly used optimal transport metric, MMD is both easier to optimize, and faster. We choose to use neural tangent kernels (NTKs) to define the kernel space where MMD operates, because of their improved descriptive power as opposed to fixed kernels. And, also, because NTKs sidestep the trivial solution, when jointly learning the inputs (video approximation) and the kernel function. Finally, we show competitive results when compared to state-of-the-art per-video methods, on six standard benchmarks. Additionally, our method has higher F1 scores than prior agglomerative work, when the number of segments is unknown.
