Logo image
Back
Optimization over Trained Neural Networks: Going Large with Gradient-Based Algorithms
Preprint   Open access

Optimization over Trained Neural Networks: Going Large with Gradient-Based Algorithms

Jiatai Tong, Yilin Zhu, Thiago Serra and Samuel Burer
ArXiv.org
Cornell University
12/30/2025
DOI: 10.48550/arxiv.2512.24295
url
https://doi.org/10.48550/arxiv.2512.24295View
Preprint (Author's original)This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

When optimizing a nonlinear objective, one can employ a neural network as a surrogate for the nonlinear function. However, the resulting optimization model can be time-consuming to solve globally with exact methods. As a result, local search that exploits the neural-network structure has been employed to find good solutions within a reasonable time limit. For such methods, a lower per-iteration cost is advantageous when solving larger models. The contribution of this paper is two-fold. First, we propose a gradient-based algorithm with lower per-iteration cost than existing methods. Second, we further adapt this algorithm to exploit the piecewise-linear structure of neural networks that use Rectified Linear Units (ReLUs). In line with prior research, our methods become competitive with -- and then dominant over -- other local search methods as the optimization models become larger.
 Mathematics - Optimization and Control

Details

Metrics

3 Record Views
Logo image