Search papers, labs, and topics across Lattice.
This paper introduces a novel frugal Neural Architecture Search (NAS) framework that integrates a Transformer controller with an Artificial Bee Colony algorithm to optimize deep learning model design on consumer-grade hardware. By employing a dynamic entropy mechanism to encourage exploration and prevent premature convergence, the method effectively addresses the cold-start problem and mitigates model bloat through algorithmic penalties on network depth. Evaluated on CIFAR-10 and credit card fraud detection tasks, the framework achieves competitive accuracy with significantly fewer parameters, demonstrating its potential for efficient model deployment in resource-constrained environments.
Achieving 84.85% accuracy with only 174,000 parameters on CIFAR-10, this framework makes advanced NAS accessible on consumer hardware.
Neural Architecture Search (NAS) has automated the design of deep learning models but traditionally requires massive computational resources, often measured in thousands of GPU-days. In this paper, we propose a frugal and memetic NAS framework designed to democratize architecture design on consumer-grade hardware. Our approach combines the global macro-search capabilities of an autoregressive Transformer controller, trained via Reinforcement Learning (RL), with the local micro-exploitation of an Artificial Bee Colony (ABC) algorithm. To prevent premature convergence during the RL phase, we introduce a dynamic entropy mechanism that forces topological exploration upon detection of performance stagnation. Evaluated on a standard GPU (NVIDIA RTX 3060), our hybrid method effectively resolves the "cold-start" problem inherent in metaheuristics. By algorithmically penalizing network depth, our framework actively mitigates model bloat: on the CIFAR-10 dataset, it discovers an efficient architecture reaching 84.85% accuracy with only $\sim$174,000 parameters (significantly smaller than standard baselines like ResNet-20) in 3 hours of search time. Furthermore, we demonstrate the framework's flexibility by applying it to credit card fraud detection, directly optimizing the F1-Score on highly imbalanced tabular data to reach a F1-Score of 0.71 with a compact network of $\sim$4,600 parameters. These results suggest that our approach can yield tailored, accessible, and highly parameter-efficient deep learning models suitable for edge deployment.
