Logo image
Back
A statistical perspective on algorithm unrolling models for inverse problems
Preprint   Open access

A statistical perspective on algorithm unrolling models for inverse problems

Yves Atchade, Xinru Liu and Qiuyun Zhu
ArXiv.org
Cornell University
11/10/2023
DOI: 10.48550/arxiv.2311.06395
url
https://doi.org/10.48550/arxiv.2311.06395View
Preprint (Author's original)This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

We consider inverse problems where the conditional distribution of the observation {\bf y} given the latent variable of interest {\bf x} (also known as the forward model) is known, and we have access to a data set in which multiple instances of {\bf x} and {\bf y} are both observed. In this context, algorithm unrolling has become a very popular approach for designing state-of-the-art deep neural network architectures that effectively exploit the forward model. We analyze the statistical complexity of the gradient descent network (GDN), an algorithm unrolling architecture driven by proximal gradient descent. We show that the unrolling depth needed for the optimal statistical performance of GDNs is of order \log(n)/\log(\varrho_n^{-1}), where n is the sample size, and \varrho_n is the convergence rate of the corresponding gradient descent algorithm. We also show that when the negative log-density of the latent variable {\bf x} has a simple proximal operator, then a GDN unrolled at depth D' can solve the inverse problem at the parametric rate O(D'/\sqrt{n}). Our results thus also suggest that algorithm unrolling models are prone to overfitting as the unrolling depth D' increases. We provide several examples to illustrate these results.
Computer Science - Learning Statistics - Machine Learning

Details

Metrics

5 Record Views
Logo image