Search papers, labs, and topics across Lattice.
This paper introduces CERF, a novel framework designed for communication-efficient and retraining-free collaborative perception among heterogeneous agents. By leveraging a new virtual modality called Poture and employing a Kalman-filter based tracker, CERF significantly reduces communication overhead by 95% while maintaining performance comparable to existing methods. The framework also allows for the seamless integration of unknown agents, enhancing its practicality for real-world applications.
Achieving a 95% reduction in communication overhead, CERF enables seamless collaboration among heterogeneous agents without the need for retraining.
Collaborative perception shares information among multiple agents to obtain a comprehensive scene representation, enhancing the perceptual capability of individual agents. However, most existing methods rely on transmitting and fusing dense feature maps for collaboration, which incurs inevitable communication overhead and heterogeneity challenges, limiting their practicality for real-world deployment. To address these challenges, we propose CERF, a novel Communication-Efficient and Retraining-Free framework for open heterogeneous collaborative perception. In CERF, we introduce a new virtual modality (termed Poture), which is generated from the perception outputs of other agents, to augment the extracted Bird's Eye View (BEV) features of the ego agent. To mitigate transmission delays, we employ a Kalman-filter based tracker and a motion forecasting model to derive the current predictions from historical perception results. Extensive experiments demonstrate that CERF achieves performance comparable to mainstream intermediate-collaboration methods while reducing communication overhead by 95% across various downstream tasks. Furthermore, CERF enables seamless integration of unknown heterogeneous agents into the existing collaborative framework without additional retraining costs. Code is available at https://github.com/uestchjw/CERF.
