Conference proceeding
The Role of Dimensionality Reduction in Classification
PROCEEDINGS OF THE TWENTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, Vol.28(1), pp.2128-2134
AAAI Conference on Artificial Intelligence
06/21/2014
DOI: 10.1609/aaai.v28i1.8975
Abstract
Dimensionality reduction (DR) is often used as a preprocessing step in classification, but usually one first fixes the DR mapping, possibly using label information, and then learns a classifier (a filter approach). Best performance would be obtained by optimizing the classification error jointly over DR mapping and classifier (a wrapper approach), but this is a difficult non convex problem, particularly with nonlinear DR. Using the method of auxiliary coordinates, we give a simple, efficient algorithm to train a combination of nonlinear DR and a classifier, and apply it to a RBF mapping with a linear SVM. This alternates steps where we train the RBF mapping and a linear SVM as usual regression and classification, respectively, with a closed form step that coordinates both. The resulting nonlinear low-dimensional classifier achieves classification errors competitive with the state-of-the-art but is fast at training and testing, and allows the user to trade off runtime for classification accuracy easily. We then study the role of nonlinear DR in linear classification, and the interplay between the DR mapping, the number of latent dimensions and the number of classes. When trained jointly, the DR mapping takes an extreme role in eliminating variation: it tends to collapse classes in latent space, erasing all manifold structure, and lay out class centroids so they are linearly separable with maximum margin.
Details
- Title: Subtitle
- The Role of Dimensionality Reduction in Classification
- Creators
- Weiran Wang - Toyota Technological Institute at ChicagoMiguel A. Carreira-Perpinan - University of California, Merced
- Resource Type
- Conference proceeding
- Publication Details
- PROCEEDINGS OF THE TWENTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, Vol.28(1), pp.2128-2134
- Publisher
- Assoc Advancement Artificial Intelligence
- Series
- AAAI Conference on Artificial Intelligence
- DOI
- 10.1609/aaai.v28i1.8975
- ISSN
- 2159-5399
- eISSN
- 2374-3468
- Number of pages
- 7
- Grant note
- 1423515 / Div Of Information & Intelligent Systems; National Science Foundation (NSF); NSF - Directorate for Computer & Information Science & Engineering (CISE) 1423515 / Direct For Computer & Info Scie & Enginr; National Science Foundation (NSF); NSF - Directorate for Computer & Information Science & Engineering (CISE)
- Language
- English
- Date published
- 06/21/2014
- Academic Unit
- Computer Science
- Record Identifier
- 9984696717302771
Metrics
6 Record Views