Preprint
A Smoothing Algorithm for l1 Support Vector Machines
ArXiv.org
12/16/2023
DOI: 10.48550/arxiv.2401.09431
Abstract
A smoothing algorithm is presented for solving the soft-margin Support Vector Machine (SVM) optimization problem with an ℓ1 penalty. This algorithm is designed to require a modest number of passes over the data, which is an important measure of its cost for very large datasets. The algorithm uses smoothing for the hinge-loss function, and an active set approach for the ℓ1 penalty. The smoothing parameter α is initially large, but typically halved when the smoothed problem is solved to sufficient accuracy. Convergence theory is presented that shows O(1+log(1+log+(1/α))) guarded Newton steps for each value of α except for asymptotic bands α=Θ(1) and α=Θ(1/N), with only one Newton step provided ηα≫1/N, where N is the number of data points and the stopping criterion that the predicted reduction is less than ηα. The experimental results show that our algorithm is capable of strong test accuracy without sacrificing training speed.
Details
- Title: Subtitle
- A Smoothing Algorithm for l1 Support Vector Machines
- Creators
- Ibrahim EmirahmetogluJeffrey HajewskiSuely OliveiraDavid E Stewart
- Resource Type
- Preprint
- Publication Details
- ArXiv.org
- DOI
- 10.48550/arxiv.2401.09431
- ISSN
- 2331-8422
- Language
- English
- Date posted
- 12/16/2023
- Academic Unit
- Computer Science; Mathematics
- Record Identifier
- 9984548298802771
Metrics
25 Record Views