Search papers, labs, and topics across Lattice.
This paper introduces GaLe, a novel memory-efficient technique designed to deploy pretrained neural networks on resource-constrained devices without the need for retraining. By partitioning feature maps into local exact and global approximate components, GaLe maintains high accuracy while significantly reducing computational overhead. Validation on ImageNet shows that GaLe achieves up to 65% speedup and 90% RAM reduction compared to traditional patch-based inference methods, demonstrating its effectiveness across various tasks including classification and generation.
GaLe achieves exact-inference performance with a staggering 65% speedup and 90% RAM reduction, revolutionizing how we deploy models on embedded devices.
Embedded devices typically lack the resources of GPU-equipped machines, and existing inference methods suffer from either high computational overhead (patch-based) or accuracy loss (approximation-based). We propose GaLe, a memory-efficient technique that enables the deployment of pretrained networks on constrained devices without retraining. GaLe partitions feature maps into two components: a local exact (Le) representation that preserves fine details and a global approximate (Ga) representation that retains long-range dependencies. Unlike standard tiling, GaLe supports global operations and attention mechanisms found in hybrid CNN-transformer models. Validated on ImageNet, our method matches exact-inference performance while achieving up to 65% speedup and 90% RAM reduction on a Cortex-M33 compared to patch-based inference. We further demonstrate GaLe's versatility across classification, detection, and generation tasks, highlighting its potential as a foundation for resource-efficient architecture design.
