Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper introduces a deep learning super-resolution architecture designed for a low-cost SPAD array on a resource-constrained microcontroller, enabling simultaneous upscaling of low-resolution depth and intensity images. The framework achieves a $\times$4 spatial resolution enhancement, reconstructing 216 $\times$ 168 images from the original 54 $\times$ 42 measurements, while meticulously managing memory and computational constraints. By evaluating hardware-oriented backbones and optimizing for INT8 quantization and bare-metal MCU deployment, the work successfully demonstrates on-device SR inference for consumer-grade SPAD sensors.
Consumer-grade SPAD sensors can now achieve 4x super-resolution on-device, even on microcontrollers with less than 1MB of SRAM.
This work presents a super-resolution (SR) deep-learning (DL) architecture to simultaneously upscale low-resolution (LR) depth and intensity images acquired by a consumer-grade VL53L9CX time-of-flight (ToF) sensor, a compact single-photon avalanche diode (SPAD) ranging device, while targeting deployment on a resource-constrained microcontroller (MCU) with only 640 KB SRAM and 2 MB Flash. The VL53L9CX provides 54 $\times$ 42 depth and intensity measurements. The proposed framework performs $\times$4 spatial SR to reconstruct 216 $\times$ 168 depth and intensity images simultaneously. The network employs separate reconstruction branches for intensity and high-resolution (HR) depth. We train the network with efficient loss functions and investigate the performance by testing multiple compact, hardware-oriented backbones, including Spatially Adaptive Feature Modulation (SAFM), Swift Parameter-Free Attention Network (SPAN), and Residual Local Feature Network (RLFN). We evaluate the network on both synthetic and real measurements, with particular attention to constrained activation and memory storage. We export the selected network to ONNX, quantize it to INT8, generate C inference code for an STM32H563ZI Arm Cortex-M33 MCU, and integrate it with our simplified sensor's firmware. This work is the first to demonstrate a compact DL SR model on a low-cost bare-metal MCU for a consumer-grade LR SPAD sensor, integrated with customized MCU firmware to enable on-device SR inference.