Search papers, labs, and topics across Lattice.
This paper addresses the bottleneck in real-time detection pipelines by enabling concurrent execution of detection and classification models on NVIDIA Jetson DLA cores, effectively eliminating GPU fallback. The authors present a five-step methodology that includes architecture adaptation and quantization-aware training, achieving a significant accuracy recovery from implicit quantization while maintaining near-zero pipeline overhead. Validation shows that their approach allows for efficient dual-head classification alongside GPU-based detection, achieving 12.5 FPS at 1080p, which is comparable to the GPU-only throughput of 13.3 FPS.
Achieving near-zero overhead in real-time detection pipelines could revolutionize edge-deployed vision systems by enabling efficient concurrent model execution without sacrificing accuracy.
Edge-deployed vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines where a detection model identifies objects of interest and downstream classifiers provide fine-grained attribute analysis. Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow. Modern edge SoCs pair GPUs with dedicated neural accelerators (NPUs, DLAs) capable of concurrent execution, yet deploying custom models on these accelerators remains impractical due to strict operator constraints, quantization incompatibilities, and an undocumented end-to-end pipeline. We target NVIDIA Jetson DLA cores as the representative platform. We present a five-step methodology for zero GPU fallback DLA INT8 deployment of classification backbones, comprising architecture adaptation, manual dynamic range workaround to rescue TensorRT's implicit quantization (recovering 94.0% accuracy from implicit quantization's 75%) for rapid pipeline validation before explicit quantization, quantization-aware training, ONNX graph surgery for DLA compilation, and a concurrent GPU-detection/DLA-classification inference pipeline. We document nine engineering constraints with root-cause analysis and generalizable solutions. Validation on a dual-head person attribute classifier running on DLA alongside a GPU object detector on a Jetson Orin NX demonstrates near-zero pipeline overhead (12.5 vs. 13.3~FPS detector-only at 1080p), with dual-DLA scaling at no additional cost. The methodology is backbone-agnostic and generalizes to any detection-classification edge pipeline.
