Logo image
Back
The multi-armed bandit problem under the mean-variance setting
Journal article   Peer reviewed

The multi-armed bandit problem under the mean-variance setting

Hongda Hu, Arthur Charpentier, Mario Ghossoub and Alexander Schied
European journal of operational research, Vol.324(1), pp.168-182
07/01/2025
DOI: 10.1016/j.ejor.2025.03.011
url
https://doi.org/10.1016/j.ejor.2025.03.011View
Published (Version of record) Open Access

Abstract

The classical multi-armed bandit problem involves a learner and a collection of arms with unknown reward distributions. At each round, the learner selects an arm and receives new information. The learner faces a tradeoff between exploiting the current information and exploring all arms. The objective is to maximize the expected cumulative reward over all rounds. Such an objective does not involve a risk-reward tradeoff, which is fundamental in many areas of application. In this paper, we build upon Sani et al. (2012)’s extension of the classical problem to a mean–variance setting. We relax their assumptions of independent arms and bounded rewards, and we consider sub-Gaussian arms. We introduce the Risk-Aware Lower Confidence Bound algorithm to solve the problem, and study some of its properties. We perform numerical simulations to demonstrate that, in both independent and dependent scenarios, our approach outperforms the algorithm suggested by Sani et al. (2012). •This paper introduces RALCB, considering risk-return balance via mean–variance.•It relaxes independence assumptions while considering sub-Gaussian arms.•The dynamic adaptation enables horizon-flexible algorithms.•The algorithm shows superior numerical performance compared to existing ones.
 Mean–variance regret analysis Multiarmed bandits Online optimization Sub-Gaussian distribution

Details

Metrics

1 Record Views
Logo image