Logo image
Back
A statistical perspective on algorithm unrolling models for inverse problems
Journal article   Open access   Peer reviewed

A statistical perspective on algorithm unrolling models for inverse problems

Yves Atchade, Xinru Liu and Qiuyun Zhu
Journal of machine learning research, Vol.26, 298
01/01/2025
url
https://jmlr.org/papers/v26/23-1512.htmlView
Published (Version of record) Open Access

Abstract

We consider inverse problems where the forward model, that is the conditional distribution of the observation y is an element of R-y(d) given the latent variable of interest x is an element of R-x(d) is known, and access is given to a data set in which multiple instances of (x,y) are observed. In this context, algorithm unrolling has become a very popular approach for designing state-of-the-art deep neural network architectures that effectively exploit the forward model. We analyze the statistical properties of the gradient descent network (GDN), a well-known architecture driven by proximal gradient descent that epitomizes unrolling learning. Under some regularity conditions, we show that when d(y)>= d(x), the GDN estimator solves the inverse problem at a statistical rate faster than the nonparametric minimax rate achievable while ignoring the forward model. Furthermore, when the negative log-density of the latent variable x has a simple proximal operator, we show that GDN achieves the parametric rate O(1/root n). Furthermore, our results are explicit in the unrolling depth of the network and suggest that unrolling models are typically prone to overfitting as the unrolling depth increases, and careful tuning as function of the sample size is required for best performances. We provide several examples to illustrate these results.
 Computer Science Technology Automation & Control Systems Computer Science, Artificial Intelligence Science & Technology

Details

Metrics

1 Record Views
Logo image