Journal article
An algorithm for distributed Bayesian inference
Stat (International Statistical Institute), Vol.11(1), e432
12/01/2022
DOI: 10.1002/sta4.432
Abstract
Monte Carlo algorithms, such as Markov chain Monte Carlo (MCMC) and Hamiltonian Monte Carlo (HMC), are routinely used for Bayesian inference; however, these algorithms are prohibitively slow in massive data settings because they require multiple passes through the full data in every iteration. Addressing this problem, we develop a scalable extension of these algorithms using the divide-and-conquer (D&C) technique that divides the data into a sufficiently large number of subsets, draws parameters in parallel on the subsets using a powered likelihood and produces Monte Carlo draws of the parameter by combining parameter draws obtained from each subset. The combined parameter draws play the role of draws from the original sampling algorithm. Our main contributions are twofold. First, we demonstrate through diverse simulated and real data analyses focusing on generalized linear models (GLMs) that our distributed algorithm delivers comparable results as the current state-of-the-art D&C algorithms in terms of statistical accuracy and computational efficiency. Second, providing theoretical support for our empirical observations, we identify regularity assumptions under which the proposed algorithm leads to asymptotically optimal inference. We also provide illustrative examples focusing on normal linear and logistic regressions where parts of our D&C algorithm are analytically tractable.
Details
- Title: Subtitle
- An algorithm for distributed Bayesian inference
- Creators
- Nariankadu D Shyamalkumar - Univ Iowa, Dept Stat & Actuarial Sci, Iowa City, IA 52242 USASanvesh Srivastava - Univ Iowa, Dept Stat & Actuarial Sci, Iowa City, IA 52242 USA
- Resource Type
- Journal article
- Publication Details
- Stat (International Statistical Institute), Vol.11(1), e432
- Publisher
- Wiley
- DOI
- 10.1002/sta4.432
- ISSN
- 2049-1573
- eISSN
- 2049-1573
- Number of pages
- 14
- Grant note
- DMS-1854667/1854662 / National Science Foundation; National Science Foundation (NSF) ONR-BAA N000141812741 / Office of Naval Research; Office of Naval Research
- Language
- English
- Date published
- 12/01/2022
- Academic Unit
- Statistics and Actuarial Science
- Record Identifier
- 9984257596902771
Metrics
15 Record Views