Search papers, labs, and topics across Lattice.
This paper addresses the challenges of bird's eye view (BEV) segmentation in autonomous driving by introducing a transformer-based variational flow transformation network (TVB) that effectively fuses data from multiple camera sensors. By employing a conditional variational autoencoder and integrating normalizing flows, the model generates multiple BEV map candidates, enhancing the realism and expressiveness of the segmentation outputs. Experimental results on nuScenes and OPV2V datasets show that TVB significantly outperforms existing methods in multi-camera view BEV segmentation and lane environment perception.
TVB achieves unprecedented accuracy in BEV segmentation by leveraging variational inference and attention mechanisms to fuse multi-camera data effectively.
The bird's eye view (BEV) has emerged as a pivotal approach for environmental perception in autonomous driving, providing a unified spatial representation for vehicles. Nevertheless, despite BEV's significance in addressing the challenges inherent to autonomous driving, effectively fusing data from multiple camera sensors and operating in complex external driving environments remains a considerable challenge. To mitigate this issue, we recast the BEV segmentation problem within a variational inference framework. In this paper, we propose a novel transformer-based variational flow transformation network for BEV segmentation, denoted as TVB. Our architecture implicitly learns the mapping from multiple camera views to a unified canonical BEV map during training by exploiting posterior BEV supervision. TVB employs a conditional variational auto encoder (CVAE) as its backbone and produces multiple BEV map candidates. To augment the realism of the generated BEV maps, we integrate normalizing flows into the map generation process, enabling the construction of more complex and expressive probability distributions. Furthermore, we design a BEV-attention fusion (BAF) module that harnesses attention mechanisms to adaptively integrate the multiple candidate BEV maps. Experimental results, evaluated on both the nuScenes and OPV2Vdatasets, demonstrate that our proposed method achieves superior performance in multi-camera view BEV segmentation and lane environment perception.
