Conference proceeding
Optimal Compression for Minimizing Classification Error Probability: An Information-Theoretic Approach
ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1-5
06/04/2023
DOI: 10.1109/ICASSP49357.2023.10096030
Abstract
We formulate the problem of performing optimal data compression under the constraints that compressed data can be used for accurate classification in machine learning. We show that this translates to a problem of minimizing the mutual information between data and its compressed version under the constraint on error probability of classification is small when using the compressed data for machine learning. We then provide analytical and computational methods to characterize the optimal trade-off between data compression and classification error probability. First, we provide an analytical characterization for the optimal compression strategy for data with binary labels. Second, for data with multiple labels, we formulate a set of convex optimization problems to characterize the optimal tradeoff, from which the optimal trade-off between the classification error and compression efficiency can be obtained by numerically solving the formulated optimization problems. We further show the improvement of our formulations over the information-bottleneck methods in classification performance.
Details
- Title: Subtitle
- Optimal Compression for Minimizing Classification Error Probability: An Information-Theoretic Approach
- Creators
- Jingchao Gao - University of IowaAo Tang - Cornell UniversityWeiyu Xu - University of Iowa
- Resource Type
- Conference proceeding
- Publication Details
- ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1-5
- Publisher
- IEEE
- DOI
- 10.1109/ICASSP49357.2023.10096030
- eISSN
- 2379-190X
- Language
- English
- Date published
- 06/04/2023
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9984503054002771
Metrics
2 Record Views