Logo image
Back
Optimization over Trained Neural Networks: Going Large with Gradient-Based Algorithms
Book chapter

Optimization over Trained Neural Networks: Going Large with Gradient-Based Algorithms

Jiatai Tong, Yilin Zhu, Thiago Serra and Samuel Burer
Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pp.596-613
Lecture Notes in Computer Science, v. 16595, Springer Nature Switzerland
2026
DOI: 10.1007/978-3-032-27242-3_35

View Online

Abstract

When optimizing a nonlinear objective, one can employ a neural network as a surrogate for the nonlinear function. However, the resulting optimization model can be time-consuming to solve globally with exact methods. As a result, local search that exploits the neural-network structure has been employed to find good solutions within a reasonable time limit. For such methods, a lower per-iteration cost is advantageous when solving larger models. The contribution of this paper is two-fold. First, we propose a gradient-based algorithm with lower per-iteration cost than existing methods. Second, we further adapt this algorithm to exploit the piecewise-linear structure of neural networks that use Rectified Linear Units (ReLUs). In line with prior research, our methods become competitive with—and then dominant over—other local search methods as the optimization models become larger.
 Constraining learning Gradient ascent Linear regions Piecewise-linear functions Rectified linear units

Details

Metrics

1 Record Views
Logo image