Conference proceeding
When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162
Proceedings of Machine Learning Research
01/01/2022
Abstract
In this paper, we propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) maximization that are applicable to deep learning. We propose new formulations of pAUC surrogate objectives by using the distributionally robust optimization (DRO) to define the loss for each individual positive data. We consider two formulations of DRO, one of which is based on conditional-value-at-risk (CVaR) that yields a non-smooth but exact estimator for pAUC, and another one is based on a KL divergence regularized DRO that yields an inexact but smooth (soft) estimator for pAUC. For both one-way and two-way pAUC maximization, we propose two algorithms and prove their convergence for optimizing their two formulations, respectively. Experiments demonstrate the effectiveness of the proposed algorithms for pAUC maximization for deep learning on various datasets. The proposed methods are implemented with tutorials in our open-sourced library LibAUC (www.libauc.org).
Details
- Title: Subtitle
- When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
- Creators
- Dixian Zhu - University of IowaGang Li - Univ Iowa, Dept Comp Sci, Ames, IA 50011 USABokun Wang - Univ Iowa, Dept Comp Sci, Ames, IA 50011 USAXiaodong Wu - University of IowaTianbao Yang - Univ Iowa, Dept Comp Sci, Ames, IA 50011 USA
- Contributors
- K Chaudhuri (Editor)S Jegelka (Editor)L Song (Editor)C Szepesvari (Editor)G Niu (Editor)S Sabato (Editor)
- Resource Type
- Conference proceeding
- Publication Details
- INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162
- Publisher
- JMLR-JOURNAL MACHINE LEARNING RESEARCH
- Series
- Proceedings of Machine Learning Research
- ISSN
- 2640-3498
- Number of pages
- 26
- Grant note
- 1844403 / NSF Career Award; National Science Foundation (NSF); NSF - Office of the Director (OD) 2110545; 1933212; CCF-1733742 / NSF; National Science Foundation (NSF)
- Language
- English
- Date published
- 01/01/2022
- Academic Unit
- The Iowa Institute for Biomedical Imaging; Electrical and Computer Engineering; Computer Science; Radiation Oncology
- Record Identifier
- 9984380464002771
Metrics
5 Record Views