Optimization over Trained Neural Networks: Taking a Relaxing Walk

Jiatai Tong; Junyang Cai; Thiago Serra

doi:10.1007/978-3-031-60599-4_14

Back

Conference proceeding

Optimization over Trained Neural Networks: Taking a Relaxing Walk

Jiatai Tong, Junyang Cai and Thiago Serra

Integration of Constraint Programming, Artificial Intelligence, and Operations Research, Vol.14743, pp.221-233

Lecture Notes in Computer Science

05/25/2024

DOI: 10.1007/978-3-031-60599-4_14

View Online

Abstract

Besides training, mathematical optimization is also used in deep learning to model and solve formulations over trained neural networks for purposes such as verification, compression, and optimization with learned constraints. However, solving these formulations soon becomes difficult as the network size grows due to the weak linear relaxation and dense constraint matrix. We have seen improvements in recent years with cutting plane algorithms, reformulations, and an heuristic based on Mixed-Integer Linear Programming (MILP). In this work, we propose a more scalable heuristic based on exploring global and local linear relaxations of the neural network model. Our heuristic is competitive with a state-of-the-art MILP solver and the prior heuristic while producing better solutions with increases in input, depth, and number of neurons.

Deep Learning

Linear Regions

Mixed-Integer Linear Programming

Neural Surrogate Models

Rectified Linear Units

Details

Title: Subtitle: Optimization over Trained Neural Networks: Taking a Relaxing Walk
Creators: Jiatai Tong - Bucknell University
Junyang Cai - Bucknell University
Thiago Serra - Bucknell University
Contributors: Bistra Dilkina (Editor)
Resource Type: Conference proceeding
Publication Details: Integration of Constraint Programming, Artificial Intelligence, and Operations Research, Vol.14743, pp.221-233
Series: Lecture Notes in Computer Science
DOI: 10.1007/978-3-031-60599-4_14
eISSN: 1611-3349
ISSN: 0302-9743
eISSN: 1611-3349
Publisher: Springer Nature Switzerland; Cham
Grant note: National Science Foundation (NSF): IIS 2104583
The authors were supported by the National Science Foundation (NSF) grant IIS 2104583, including Junyang Cai while at Bucknell.
Language: English
Date published: 05/25/2024
Academic Unit: Business Analytics
Record Identifier: 9984696756402771

Metrics

7 Record Views

2 Times Cited - Web of Science