Search papers, labs, and topics across Lattice.
The paper introduces Articulation in Motion (AiM), a novel framework for analyzing articulated objects from a user-object interaction video and a start-state scan, without requiring prior knowledge of the number of parts. AiM employs a dual-Gaussian scene representation learned from the initial scan and video, leveraging motion cues for part segmentation and joint assignment. The method uses a sequential RANSAC approach for part mobility analysis, clustering moving primitives into rigid parts and estimating kinematics while automatically determining the number of parts, resulting in high-quality rendering and part segmentation.
Discovering articulation joints and part mobility from video just got easier: AiM needs no prior knowledge of part count or structure.
Articulated objects are ubiquitous in daily life. Our goal is to achieve a high-quality reconstruction, segmentation of independent moving parts, and analysis of articulation. Recent methods analyse two different articulation states and perform per-point part segmentation, optimising per-part articulation using cross-state correspondences, given a priori knowledge of the number of parts. Such assumptions greatly limit their applications and performance. Their robustness is reduced when objects cannot be clearly visible in both states. To address these issues, in this paper, we present a new framework, Articulation in Motion (AiM). We infer part-level decomposition, articulation kinematics, and reconstruct an interactive 3D digital replica from a user-object interaction video and a start-state scan. We propose a dual-Gaussian scene representation that is learned from an initial 3DGS scan of the object and a video that shows the movement of separate parts. It uses motion cues to segment the object into parts and assign articulation joints. Subsequently, a robust, sequential RANSAC is employed to achieve part mobility analysis without any part-level structural priors, which clusters moving primitives into rigid parts and estimates kinematics while automatically determining the number of parts. The proposed approach separates the object into parts, each represented as a 3D Gaussian set, enabling high-quality rendering. Our approach yields higher quality part segmentation than previous methods, without prior knowledge. Extensive experimental analysis on both simple and complex objects validates the effectiveness and strong generalisation ability of our approach. Project page: https://haoai-1997.github.io/AiM/.
