Search papers, labs, and topics across Lattice.
The paper introduces Neurosim, a high-performance simulator capable of rendering dynamic vision sensors, RGB cameras, depth sensors, inertial sensors, and agile multi-rotor vehicle dynamics at frame rates up to 2700 FPS on a desktop GPU. Neurosim is designed to address the need for fast and realistic sensor simulation for training and testing neuromorphic perception and control algorithms. The simulator integrates with Cortex, a ZeroMQ-based communication library, to facilitate seamless integration with machine learning frameworks like PyTorch and robotics workflows, enabling self-supervised learning and closed-loop testing.
Achieve near real-time simulation of neuromorphic sensors and robot dynamics, enabling faster training and testing of perception and control algorithms.
Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .
