Conference proceeding
Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scaling
Proceedings of the ... AAAI Conference on Artificial Intelligence, Vol.36(7), pp.7381-7389
AAAI Conference on Artificial Intelligence
06/30/2022
DOI: 10.1609/aaai.v36i7.20701
Abstract
We develop a new method of online inference for a vector of parameters estimated by the Polyak-Ruppert averaging procedure of stochastic gradient descent (SGD) algorithms. We leverage insights from time series regression in econometrics and construct asymptotically pivotal statistics via random scaling. Our approach is fully operational with online data and is rigorously underpinned by a functional central limit theorem. Our proposed inference method has a couple of key advantages over the existing methods. First, the test statistic is computed in an online fashion with only SGD iterates and the critical values can be obtained without any resampling methods, thereby allowing for efficient implementation suitable for massive online data. Second, there is no need to estimate the asymptotic variance and our inference method is shown to be robust to changes in the tuning parameters for SGD algorithms in simulation experiments with synthetic data.
Details
- Title: Subtitle
- Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scaling
- Creators
- Sokbae Lee - Columbia UniversityYuan Liao - Rutgers, The State University of New JerseyMyung Hwan Seo - Seoul National UniversityYoungki Shin - McMaster University
- Resource Type
- Conference proceeding
- Publication Details
- Proceedings of the ... AAAI Conference on Artificial Intelligence, Vol.36(7), pp.7381-7389
- Series
- AAAI Conference on Artificial Intelligence
- DOI
- 10.1609/aaai.v36i7.20701
- ISSN
- 2159-5399
- eISSN
- 2374-3468
- Publisher
- Assoc Advancement Artificial Intelligence
- Number of pages
- 9
- Grant note
- ES/P008909/1 / UK Economic and Social Research Council (ESRC); UK Research & Innovation (UKRI); Economic & Social Research Council (ESRC) McMaster COVID-19 Research Fund Korea Bureau of Economic Research in the Institute of Economic Research of Seoul National University ERC-2014-CoG-646917ROMIA / European Research Council; European Research Council (ERC) NRF-2018S1A5A2A01033487 / National Research Foundation of Korea Compute Ontario Ministry of Education of the Republic of Korea; Ministry of Education (MOE), Republic of Korea Compute Canada
- Language
- English
- Date published
- 06/30/2022
- Academic Unit
- Economics
- Record Identifier
- 9984936836902771
Metrics
7 Record Views