Search papers, labs, and topics across Lattice.
This paper introduces S3KF, a novel panoramic 3D multi-object tracking framework designed for wide-area robotic perception using a rotating LiDAR and quad-fisheye camera rig. S3KF represents object bearing on a unit sphere using a tangent-plane parameterization, jointly estimating it with box scale and depth dynamics via an extended spherical Kalman filter. Experiments on real-world sequences demonstrate decimeter-level planar tracking accuracy and improved identity continuity, achieving real-time performance on a Jetson AGX Orin.
Achieve robust, real-time 3D multi-object tracking in panoramic views by representing object states on a sphere, sidestepping the limitations of image-plane trackers and redundant Euclidean formulations.
Panoramic multi-object tracking is important for industrial safety monitoring, wide-area robotic perception, and infrastructure-light deployment in large workspaces. In these settings, the sensing system must provide full-surround coverage, metric geometric cues, and stable target association under wide field-of-view distortion and occlusion. Existing image-plane trackers are tightly coupled to the camera projection and become unreliable in panoramic imagery, while conventional Euclidean 3D formulations introduce redundant directional parameters and do not naturally unify angular, scale, and depth estimation. In this paper, we present $\mathbf{S^3KF}$, a panoramic 3D multi-object tracking framework built on a motorized rotating LiDAR and a quad-fisheye camera rig. The key idea is a geometry-consistent state representation on the unit sphere $\mathbb{S}^2$, where object bearing is modeled by a two-degree-of-freedom tangent-plane parameterization and jointly estimated with box scale and depth dynamics. Based on this state, we derive an extended spherical Kalman filtering pipeline that fuses panoramic camera detections with LiDAR depth observations for multimodal tracking. We further establish a map-based ground-truth generation pipeline using wearable localization devices registered to a shared global LiDAR map, enabling quantitative evaluation without motion-capture infrastructure. Experiments on self-collected real-world sequences show decimeter-level planar tracking accuracy, improved identity continuity over a 2D panoramic baseline in dynamic scenes, and real-time onboard operation on a Jetson AGX Orin platform. These results indicate that the proposed framework is a practical solution for panoramic perception and industrial-scale multi-object tracking.The project page can be found at https://kafeiyin00.github.io/S3KF/.
