Search papers, labs, and topics across Lattice.
This paper introduces NightSight, a lightweight perception system for autonomous navigation in complete darkness using a monocular event camera, a coded aperture lens, and an infrared dot projector. The system decodes depth-dependent blur signatures created by the projected pattern and coded aperture into dense depth maps using a CNN trained solely on synthetic data. The system achieves high accuracy (7.0cm L1 error up to 2.5m) in real-world scenes while running at 20 Hz on an NVIDIA Jetson Orin Nano, demonstrating its suitability for resource-constrained robots.
Zero-shot depth estimation in complete darkness is now possible on resource-constrained robots, thanks to a clever combination of event cameras, coded apertures, and synthetic training.
Small aerial robots are particularly well-suited for search and rescue in confined and hazardous environments due to their agility, low cost, and ability to traverse through cluttered spaces that are inaccessible to larger platforms. However, enabling autonomous navigation in complete darkness remains a significant challenge, because small aerial robots cannot easily accommodate perception systems that demand substantial payload, power, or computation. In this work, we present a lightweight perception approach that combines a monocular event camera, a coded aperture lens, and an infrared dot projector to enable navigation in such conditions. The projected pattern, when imaged through the coded aperture, produces depth dependent blur signatures that implicitly encode scene geometry. We train a convolutional neural network to decode these signatures into dense depth maps using only synthetic data generated from a simple planar wall setup. Despite this minimal training regime, the model generalizes zero-shot to complex real-world scenes. Our system operates in real time at 20 Hz on a NVIDIA Jetson Orin Nano, demonstrating suitability for resource-constrained platforms. We further analyze the impact of different coded aperture designs on depth estimation performance. Our approach gives high accuracy (l1 error 7.0cm) upto 2.5m range (2.80% error). These results highlight the potential of combining structured illumination, coded optics, and event-based sensing for enabling robust perception and navigation in complete darkness.
