Conference proceeding
Smoothed Hinge Loss and ℓ1 Support Vector Machines
2018 IEEE International Conference on Data Mining Workshops (ICDMW), Vol.2018-, pp.1217-1223
11/2018
DOI: 10.1109/ICDMW.2018.00174
Abstract
A standard approach to solving the Support Vector Machine (SVM) optimization problem is to solve the dual problem, typically using a coordinate descent algorithm. When solving the dual problem, however, the compute load increases with the number of data points. Some approaches, such as Pegasos [1] have had great success solving the primal problem, rather than the dual. We follow the idea of solving the primal, rather than dual, problem but introduce a new algorithm that makes minimal use of the gradient and uses approximate second-order information in the form of an approximate Hessian. In particular, we present a new algorithm for solving the soft-margin SVM optimization problem with an ℓ^1 penalty, called SmSVM. This algorithm is designed to require a modest number of passes over the data and can achieve an optimal solution with a minimal number of gradient calculations. The algorithm uses a smooth approximation for the hinge-loss function, and an active set approach for the ℓ^1 penalty. We use the active set approach to make implementation optimizations by taking advantage of the feature selection to reduce the problem size of our matrix-vector and vector-vector linear algebra operations. These optimizations result in substantially faster training times, in addition to providing feature selection. Our experiments show impressive results, beating LIBLINEAR [2] in many scenarios both in test accuracy and training time. Our solutions also benefit from sparsity, thanks to the ℓ^1 regularizer.
Details
- Title: Subtitle
- Smoothed Hinge Loss and ℓ1 Support Vector Machines
- Creators
- Jeffrey HajewskiSuely OliveiraDavid Stewart
- Resource Type
- Conference proceeding
- Publication Details
- 2018 IEEE International Conference on Data Mining Workshops (ICDMW), Vol.2018-, pp.1217-1223
- Publisher
- IEEE
- DOI
- 10.1109/ICDMW.2018.00174
- ISSN
- 2375-9232
- eISSN
- 2375-9259
- Language
- English
- Date published
- 11/2018
- Academic Unit
- Computer Science; Mathematics
- Record Identifier
- 9983985700902771
Metrics
20 Record Views