Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Edge-TSR, a continuous edge inference system designed for roadside perception on the NVIDIA Jetson Orin Nano, addressing the limitations of traditional benchmark evaluations that overlook deployment-specific challenges. The authors find that real-world streaming deployments can lead to a 20-30% drop in performance compared to static-image benchmarks, highlighting the critical need for deployment-aware evaluation. By integrating a lightweight track-aware temporal stabilization mechanism, Edge-TSR recovers up to 10.16% in classification accuracy while maintaining real-time performance during sustained operation, demonstrating its effectiveness in practical scenarios.
Benchmark evaluations can mislead performance expectations by overstating edge inference capabilities, revealing a 20-30% performance drop in real-world applications.
Continuous AI inference on resource-constrained edge hardware introduces deployment effects that are largely invisible to conventional benchmark evaluation, including temporal instability in streaming video, thermal throttling under sustained load, and workload-dependent performance variability. We present Edge-TSR, a deployment-oriented continuous edge inference system for sustained roadside perception on the NVIDIA Jetson Orin Nano. Edge-TSR integrates detection, tracking, fine-grained classification, and a lightweight track-aware temporal stabilization mechanism that improves streaming inference consistency with negligible computational overhead. Our central finding is that benchmark-centric evaluation systematically overstates deployed edge inference performance. Across three state-of-the-art baselines, we observe consistent 20-30% relative degradation when transitioning from static-image evaluation to real-world streaming deployment. Edge-TSR addresses this gap through temporal inference stabilization, recovering up to 10.16% classification accuracy over per-frame inference baselines while maintaining sustained real-time performance under continuous operation. We evaluate the complete system under diverse real-world deployment conditions, jointly characterizing inference quality, latency, throughput, and thermal behavior during long-duration operation. A 55-minute vehicular deployment over a 26 km route demonstrates sustained operation at 16.18 FPS within safe thermal limits on a single embedded device without cloud offload. Our findings show that deployment-aware evaluation and temporal inference stabilization are necessary components of continuously operating edge AI systems intended for real-world sensing deployments. We release a sample annotated streaming video evaluation dataset and full system implementation to support reproducible deployment-centric evaluation.