Shallow Neural Network Training Via Atomic Norms and Semidefinite Programming

Andrew J. Christensen; Ananya Sen Gupta

doi:10.1109/LSP.2025.3643361

Back

Shallow Neural Network Training Via Atomic Norms and Semidefinite Programming

Journal article

Peer reviewed

Shallow Neural Network Training Via Atomic Norms and Semidefinite Programming

Andrew J. Christensen and Ananya Sen Gupta

IEEE signal processing letters, Vol.33(325), p.321

2026

DOI: 10.1109/LSP.2025.3643361

View Online

Abstract

Neural networks have achieved remarkable results across numerous scientific domains because of their ability to uncover complex patterns. However, despite their effectiveness, these networks rely on heuristic training of highly non-convex objective functions, limiting theoretical understanding and practical reliability. Recent work has shown that shallow neural networks with scalar outputs can be formulated as convex optimization problems, bridging empirical success with theory. In this work, we build upon this framework for vector-valued outputs, introducing a convex formulation for two-layer ReLU networks based on an atomic norm and expressible as a semidefinite program (SDP). This yields a principled convex relaxation of multi-output networks that is both expressive and tractable. We validate the approach using standard SDP solvers, demonstrating its feasibility. These results extend convex neural network training beyond scalar outputs and provide a foundation for scalable, robust alternatives to current heuristic deep learning methods. Our method achieved a 7.3% increase in classification accuracy compared to a baseline convex multi-output network.

Neurons

Optimization

Atomic layer deposition

Atomic Norm

Atoms

Biological neural networks

Convex Optimization

Neural Networks

Semidefinite Programming

Symmetric matrices

Training

Urban areas

Vectors

Details

Title: Subtitle: Shallow Neural Network Training Via Atomic Norms and Semidefinite Programming
Creators: Andrew J. Christensen - University of Iowa
Ananya Sen Gupta - University of Iowa
Resource Type: Journal article
Publication Details: IEEE signal processing letters, Vol.33(325), p.321
DOI: 10.1109/LSP.2025.3643361
ISSN: 1070-9908
eISSN: 1558-2361
Publisher: IEEE
Grant note: Department of Defense Navy (NEEC): N00174201001 Office of Naval Research: N000142312503
This work was supported in part by the Office of Naval Research under Grant N000142312503 and in part by the Department of Defense Navy (NEEC) under Grant N00174201001.
Language: English
Electronic publication date: 12/11/2025
Date published: 2026
Academic Unit: Electrical and Computer Engineering
Record Identifier: 9985091815102771

Metrics

2 Record Views