Conference proceeding
Stochastic Optimization for Non-convex Inf-Projection Problems
Proceedings of Machine Learning Research, Vol.119, pp.10660-10669
Proceedings of the International Conference on Machine Learning, 37th (07/2020)
2020
Abstract
In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable. This problem include difference of convex (DC) functions and a family of bi-convex functions as special cases. We develop stochastic algorithms and establish their first-order convergence for finding a (nearly) stationary solution of the target non-convex function under different conditions of the component functions. To the best of our knowledge, this is the first work that comprehensively studies stochastic optimization of non-convex inf-projection minimization problems with provable convergence guarantee. Our algorithms enable efficient stochastic optimization of a family of non-decomposable DC functions and a family of bi-convex functions. To demonstrate the power of the proposed algorithms we consider an important application in variance-based regularization. Experiments verify the effectiveness of our inf-projection based formulation and the proposed stochastic algorithm in comparison with previous stochastic algorithms based on the min-max formulation for achieving the same effect.
Details
- Title: Subtitle
- Stochastic Optimization for Non-convex Inf-Projection Problems
- Creators
- Yan YanYi XuLijun ZhangXiaoyu WangTianbao Yang
- Resource Type
- Conference proceeding
- Publication Details
- Proceedings of Machine Learning Research, Vol.119, pp.10660-10669
- Conference
- Proceedings of the International Conference on Machine Learning, 37th (07/2020)
- Language
- English
- Date published
- 2020
- Academic Unit
- Computer Science
- Record Identifier
- 9984259424702771
Metrics
8 Record Views