TESTNAV: Pareto-Guided Search for Compositional Robustness Testing
Arif, Arooj, Hartung, Tobias, Botoeva, Elena and Koliousis, Alexandros (2026) TESTNAV: Pareto-Guided Search for Compositional Robustness Testing. In: Robustifying Generative AI for Reliable, Safe, and Human-Centric Systems, Aug 15, 2026, Bremen, Germany.
Abstract
Deep learning models remain vulnerable to real- world input perturbations, especially when multiple corruptions co-occur in the same input (e.g., bright- ness shifts and motion blur). Compositional test- ing reveals these interaction effects but introduces two challenges: combinatorial growth of the per- turbation space as dimensions and severity levels increase, and uneven diagnostic value—many com- binations yield unrealistically degraded inputs with limited practical relevance. We present TESTNAV, 1 a Pareto-guided robustness testing framework for efficiently exploring discrete, compositional perturbation spaces when only a lim- ited number of perturbation configurations can be evaluated. TESTNAV prioritises severe yet real- istic failures by formulating robustness testing as bi-objective optimisation: maximise performance degradation while preserving input fidelity mea- sured by modality-specific metrics (e.g., SSIM and KID for vision; chrF and BERT-F1 for language and code). It uses NSGA-II to approximate the bi-objective Pareto front. Across four benchmarks spanning vision, natural language, and code gener- ation, TESTNAV recovers Pareto fronts up to 2.15× faster than search-based baselines, using 35.8%– 89.3% of the discrete perturbation space defined by four perturbation dimensions with six levels each.
Actions (login required)
 |
Edit Item |