Search papers, labs, and topics across Lattice.
This paper introduces TESTNAV, a Pareto-guided framework designed to enhance compositional robustness testing in deep learning models by efficiently navigating the complex space of input perturbations. By framing robustness testing as a bi-objective optimization problem, TESTNAV seeks to maximize performance degradation while maintaining input fidelity, utilizing NSGA-II to approximate the Pareto front. The results show that TESTNAV can recover Pareto fronts up to 2.15 times faster than traditional search-based methods, effectively using a significant portion of the discrete perturbation space across multiple benchmarks in vision, language, and code generation.
TESTNAV achieves up to 2.15x faster exploration of compositional perturbation spaces while ensuring realistic and severe input failures are prioritized.
Deep learning models remain vulnerable to real-world input perturbations, especially when multiple corruptions co-occur in the same input (e.g., brightness shifts and motion blur). Compositional testing reveals these interaction effects but introduces two challenges: combinatorial growth of the perturbation space as dimensions and severity levels increase, and uneven diagnostic value-many combinations yield unrealistically degraded inputs with limited practical relevance. We present TESTNAV, 1 a Pareto-guided robustness testing framework for efficiently exploring discrete, compositional perturbation spaces when only a limited number of perturbation configurations can be evaluated. TESTNAV prioritises severe yet realistic failures by formulating robustness testing as bi-objective optimisation: maximise performance degradation while preserving input fidelity measured by modality-specific metrics (e.g., SSIM and KID for vision; chrF and BERT-F1 for language and code). It uses NSGA-II to approximate the bi-objective Pareto front. Across four benchmarks spanning vision, natural language, and code generation, TESTNAV recovers Pareto fronts up to 2.15x faster than search-based baselines, using 35.8%-89.3% of the discrete perturbation space defined by four perturbation dimensions with six levels each.
