Logo image
Back
Infinite-Dimensional Operator/Block Kaczmarz Algorithms: Regret Bounds andλ -Effectiveness
Preprint   Open access

Infinite-Dimensional Operator/Block Kaczmarz Algorithms: Regret Bounds andλ -Effectiveness

Halyun Jeong, Palle E. T Jorgensen, Hyun-Kyoung Kwon and Myung-Sin Song
ArXiv.org
Cornell University
11/10/2025
DOI: 10.48550/arxiv.2511.07604
url
https://doi.org/10.48550/arxiv.2511.07604View
Preprint (Author's original)This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

We present a variety of projection-based linear regression algorithms with a focus on modern machine-learning models and their algorithmic performance. We study the role of the relaxation parameter in generalized Kaczmarz algorithms and establish a priori regret bounds with explicit$λ$ -dependence to quantify how much an algorithm's performance deviates from its optimal performance. A detailed analysis of relaxation parameter is also provided. Applications include: explicit regret bounds for the framework of Kaczmarz algorithm models, non-orthogonal Fourier expansions, and the use of regret estimates in modern machine learning models, including for noisy data, i.e., regret bounds for the noisy Kaczmarz algorithms. Motivated by machine-learning practice, our wider framework treats bounded operators (on infinite-dimensional Hilbert spaces), with updates realized as (block) Kaczmarz algorithms, leading to new and versatile results.
 Computer Science - Learning Mathematics - Functional Analysis Statistics - Machine Learning

Details

Metrics

25 Record Views
Logo image