Search papers, labs, and topics across Lattice.
The paper introduces CUBE, a novel 3D face representation that uses a B-spline volume defined by a lattice of learned high-dimensional control features. This representation maps a 3D parametric domain to 3D Euclidean space through a two-stage process involving B-spline blending and an MLP-based residual displacement prediction. Experiments demonstrate that CUBE, when used as a decoder with transformer-based encoders, achieves state-of-the-art results in 3D scan registration and monocular 3D face reconstruction while retaining local editability.
Forget meshes of triangles: learned B-spline volumes offer a powerful, editable, and state-of-the-art representation for 3D faces.
We present CUBE (Control-based Unified B-spline Encoding), a new geometric representation for human faces that combines B-spline volumes with learned features, and demonstrate its use as a decoder for 3D scan registration and monocular 3D face reconstruction. Unlike existing B-spline representations with 3D control points, CUBE is parametrized by a lattice (e.g., 8 x 8 x 8) of high-dimensional control features, increasing the model's expressivity. These features define a continuous, two-stage mapping from a 3D parametric domain to 3D Euclidean space via an intermediate feature space. First, high-dimensional control features are locally blended using the B-spline bases, yielding a high-dimensional feature vector whose first three values define a 3D base mesh. A small MLP then processes this feature vector to predict a residual displacement from the base shape, yielding the final refined 3D coordinates. To reconstruct 3D surfaces in dense semantic correspondence, CUBE is queried at 3D coordinates sampled from a fixed template mesh. Crucially, CUBE retains the local support property of traditional B-spline representations, enabling local surface editing by updating individual control features. We demonstrate the strengths of this representation by training transformer-based encoders to predict CUBE's control features from unstructured point clouds and monocular images, achieving state-of-the-art scan registration results compared to recent baselines.
