Search papers, labs, and topics across Lattice.
This paper introduces EsaacSim, a multimodal event camera add-on for NVIDIA Isaac Sim that facilitates the online simulation of configurable event cameras, generating both grayscale and Bayer RGGB events. The framework allows for synchronized outputs across various modalities, including RGB, depth, and IMU, and employs a motion-guided frame-gap synthesis strategy to enhance temporal resolution while maintaining compatibility with existing simulation pipelines. Experimental results indicate efficient performance across multiple resolutions, demonstrating EsaacSim's capability to support advanced robotics research and synthetic data generation with effective event rates ranging from 240 to 960Hz.
EsaacSim achieves synchronized multimodal event-camera simulation, pushing the boundaries of robotics research with effective event rates up to 960Hz.
Event-based vision is becoming an increasingly important sensing paradigm for robotics, yet its adoption remains limited by sensor availability and the lack of integrated simulation tools for modern robotics platforms. This paper presents EsaacSim, a multimodal event camera add-on for NVIDIA Isaac Sim that enables online simulation of configurable event cameras with grayscale and Bayer RGGB event generation. The framework supports multiple event camera resolutions and provides synchronized RGB, APS, event, depth, and IMU outputs through native ROS2 interfaces. A motion-guided frame-gap synthesis strategy further increases the effective temporal resolution while preserving compatibility with the Isaac Sim rendering pipeline. Experimental evaluation demonstrates synchronized multimodal simulation across representative robotic scenes and efficient online performance over five event camera resolutions at effective event rates from 240 to 960Hz. Event stream generation requires 6.98--27.28ms for grayscale events and 7.58--29.16ms for Bayer RGGB events while using less than 400MB of additional GPU memory on an NVIDIA RTX~4060 GPU. These results show that EsaacSim enables supports online multimodal event-camera simulation for robotics research and synthetic data generation. We release an early version of the simulator and report its current architecture and performance.
