Search papers, labs, and topics across Lattice.
This paper introduces PANY, a model-free framework for estimating the 6D pose of unseen objects using both RGB and RGB-D inputs, which overcomes the limitations of existing methods that rely on CAD assets or single-anchor matching. By leveraging a multi-view transformer geometry backbone, PANY learns view-consistent geometry and cross-view alignment cues, enabling it to perform robustly under occlusion and significant viewpoint changes. Experimental results show that PANY achieves state-of-the-art performance, improving pose accuracy by over 12% on the YCB-V dataset and more than 20% on LM-O, demonstrating its effectiveness in real-world scenarios with sparse reference views.
PANY outperforms existing model-free methods by over 20% in pose accuracy, even in challenging conditions with limited reference overlap.
Estimating the 6D pose of unseen objects is a fundamental yet challenging problem for open-world robotics and embodied perception. Model-based methods are accurate but depend on CAD assets or heavy onboarding, while most model-free approaches are still limited to pairwise single-anchor matching and thus fail under occlusion and large viewpoint changes with low query-reference overlap. Therefore, we present PANY, a unified model-free framework that seamlessly supports both RGB and RGB-D inputs, operates on one or sparse pose-free reference views, and generalizes effectively to novel objects. Built on a multi-view transformer geometry backbone, PANY moves beyond pairwise matching by learning view-consistent geometry and cross-view alignment cues that remain stable under wide baselines and limited overlap. When additional unposed assist views are available, PANY aggregates them via pose-graph canonical registration to increase geometric coverage and reinforce the final pose. Extensive experiments show that PANY achieves state-of-the-art performance across multiple benchmarks, substantially outperforming existing model-free methods, improving pose accuracy by +12% on YCB-V and over +20% on LM-O. Furthermore, PANY consistently performs well under both single-reference and sparse-reference settings, demonstrating strong robustness in real-world environments.
