Preprint
Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Optimization with Non-Smooth Convex Constraints
ArXiv.org
01/30/2023
DOI: 10.48550/arxiv.2301.13314
Abstract
In this paper, we consider a general non-convex constrained optimization
problem, where the objective function is weakly convex and the constraint
function is convex while they can both be non-smooth. This class of problems
arises from many applications in machine learning such as fairness-aware
supervised learning. To solve this problem, we consider the classical switching
subgradient method by Polyak (1965), which is an intuitive and easily
implementable first-order method. Before this work, its iteration complexity
was only known for convex optimization. We prove its oracle complexity for
finding a nearly stationary point when the objective function is non-convex.
The analysis is derived separately when the constraint function is
deterministic and stochastic. Compared to existing methods, especially the
double-loop methods, the switching gradient method can be applied to non-smooth
problems and only has a single loop, which saves the effort on tuning the
number of inner iterations.
Details
- Title: Subtitle
- Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Optimization with Non-Smooth Convex Constraints
- Creators
- Yankun HuangQihang Lin
- Resource Type
- Preprint
- Publication Details
- ArXiv.org
- DOI
- 10.48550/arxiv.2301.13314
- ISSN
- 2331-8422
- Language
- English
- Date posted
- 01/30/2023
- Academic Unit
- Business Analytics
- Record Identifier
- 9984380585202771
Metrics
138 Record Views