Search papers, labs, and topics across Lattice.
This paper introduces InterOCF, a novel framework for camera-only 4D occupancy forecasting that enhances predictive accuracy by jointly modeling temporal dynamics in both 3D voxel representations and multi-view segmentation sequences. By integrating a 3D Spatio-Temporal module, a 2D Spatio-Temporal module, and a Spatio-Temporal Interaction Modeling module, the framework effectively captures the complex interactions between 2D and 3D data. Experimental results on multiple datasets demonstrate that InterOCF significantly outperforms existing methods, highlighting the importance of spatial-temporal modeling in autonomous vehicle applications.
InterOCF reveals that effectively integrating 2D and 3D spatio-temporal dynamics can dramatically enhance the accuracy of occupancy forecasting for autonomous vehicles.
Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images, which is critical for driving safety. Even though current methods have achieved good performance, the strong spatial-temporal modeling between the input multi-view frames is still underexplored, which limits the performance of those methods in future 4D forecasting. To address this gap, we introduce a novel framework, InterOCF, for 4D occupancy forecasting that jointly models temporal dynamics in both 3D voxel-based representations and multi-view segmentation sequences, while explicitly incorporating feature interaction between the 2D and 3D branches. Our framework incorporates three core components: 1) A 3D Spatio-Temporal (3DST) module that learns volumetric dynamics from historical voxel states to predict future voxel states; 2) A 2D Spatio-Temporal (2DST) module employing an auxiliary multi-view temporal segmentation forecasting task to enhance temporal semantic dynamics; 3) A Spatio-Temporal Interaction Modeling (STIM) module that enables feature interaction between 2D and 3D representations. Experiments on the nuScenes, Lyft-Level5, and nuScenes-Occupancy datasets show that InterOCF consistently outperforms existing baseline approaches.
