Conference proceeding
Stochastic optimization for deep CCA via nonlinear orthogonal iterations
2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp.688-695
09/01/2015
DOI: 10.1109/ALLERTON.2015.7447071
Abstract
Deep CCA is a recently proposed deep neural network extension to the traditional canonical correlation analysis (CCA), and has been successful for multi-view representation learning in several domains. However, stochastic optimization of the deep CCA objective is not straightforward, because it does not decouple over training examples. Previous optimizers for deep CCA are either batch-based algorithms or stochastic optimization using large minibatches, which can have high memory consumption. In this paper, we tackle the problem of stochastic optimization for deep CCA with small minibatches, based on an iterative solution to the CCA objective, and show that we can achieve as good performance as previous optimizers and thus alleviate the memory requirement.
Details
- Title: Subtitle
- Stochastic optimization for deep CCA via nonlinear orthogonal iterations
- Creators
- Weiran Wang - Toyota Technological Institute at ChicagoRaman Arora - Johns Hopkins UniversityKaren Livescu - Toyota Technological Institute at ChicagoNathan Srebro - Toyota Technological Institute at Chicago
- Resource Type
- Conference proceeding
- Publication Details
- 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp.688-695
- Publisher
- IEEE
- DOI
- 10.1109/ALLERTON.2015.7447071
- Language
- English
- Date published
- 09/01/2015
- Academic Unit
- Computer Science
- Record Identifier
- 9984696561002771
Metrics
1 Record Views