Search papers, labs, and topics across Lattice.
This paper introduces Adaptive Data Dropout, a novel training framework that dynamically adjusts the subset of training data used based on real-time performance feedback. Inspired by self-regulated learning, the method employs a lightweight stochastic update mechanism to modulate the dropout schedule online, balancing exploration and consolidation. Experiments on image classification benchmarks demonstrate that Adaptive Data Dropout reduces effective training steps while maintaining competitive accuracy compared to static data dropout strategies.
Training deep nets doesn't need to be a data deluge: dynamically dropping less-useful training examples during learning can maintain accuracy while slashing compute.
Deep neural networks are typically trained by uniformly sampling large datasets across epochs, despite evidence that not all samples contribute equally throughout learning. Recent work shows that progressively reducing the amount of training data can improve efficiency and generalization, but existing methods rely on fixed schedules that do not adapt during training. In this work, we propose Adaptive Data Dropout, a simple framework that dynamically adjusts the subset of training data based on performance feedback. Inspired by self-regulated learning, our approach treats data selection as an adaptive process, increasing or decreasing data exposure in response to changes in training accuracy. We introduce a lightweight stochastic update mechanism that modulates the dropout schedule online, allowing the model to balance exploration and consolidation over time. Experiments on standard image classification benchmarks show that our method reduces effective training steps while maintaining competitive accuracy compared to static data dropout strategies. These results highlight adaptive data selection as a promising direction for efficient and robust training. Code will be released.
