Search papers, labs, and topics across Lattice.
This paper introduces Natural Selection (NS), a novel optimization method that incorporates competitive interactions among samples during deep network training by assembling samples into composite images and using the resulting predictions to compute a natural selection score for each sample. These scores dynamically reweight the sample-wise loss, introducing a competition-driven mechanism. Experiments across 12 datasets and four image classification tasks demonstrate NS's effectiveness, compatibility with diverse architectures, and task-agnostic nature, showcasing its potential for adaptive and balanced model optimization.
Stop training all your data the same way: this new method adaptively reweights samples based on how "competitive" they are within a group, leading to better performance and robustness.
Conventional deep network training generally optimizes all samples under a largely uniform learning paradigm, without explicitly modeling the heterogeneous competition among them. Such an oversimplified treatment can lead to several well-known issues, including bias under class imbalance, insufficient learning of hard samples, and the erroneous reinforcement of noisy samples. In this work, we present \textit{Natural Selection} (NS), a novel evolution-inspired optimization method that explicitly incorporates competitive interactions into deep network training. Unlike conventional sample reweighting strategies that rely mainly on predefined heuristics or static criteria, NS estimates the competitive status of each sample in a group-wise context and uses it to adaptively regulate its training contribution. Specifically, NS first assembles multiple samples into a composite image and rescales it to the original input size for model inference. Based on the resulting predictions, a natural selection score is computed for each sample to characterize its relative competitive variation within the constructed group. These scores are then used to dynamically reweight the sample-wise loss, thereby introducing an explicit competition-driven mechanism into the optimization process. In this way, NS provides a simple yet effective means of moving beyond uniform sample treatment and enables more adaptive and balanced model optimization. Extensive experiments on 12 public datasets across four image classification tasks demonstrate the effectiveness of the proposed method. Moreover, NS is compatible with diverse network architectures and does not depend on task-specific assumptions, indicating its strong generality and practical potential. The code will be made publicly available.
