On the convergence of stochastic primal-dual hybrid gradient - IMT - Institut Mines-Télécom Accéder directement au contenu
Article Dans Une Revue SIAM Journal on Optimization Année : 2022

On the convergence of stochastic primal-dual hybrid gradient

Résumé

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution and linear convergence with standard step sizes, independent of strong convexity constants. Our assumption for linear convergence is metric subregularity, which is satisfied for smooth and strongly convex problems in addition to many nonsmooth and/or nonstrongly convex problems, such as linear programs, Lasso, and support vector machines. In the general convex case, we prove optimal sublinear rates for the ergodic sequence and for randomly selected iterate, without bounded domain assumptions. We also provide numerical evidence showing that SPDHG with standard step sizes shows favorable and robust practical performance against its specialized strongly convex variant SPDHG-$\mu$ and other state-of-the-art algorithms including variance reduction methods and stochastic dual coordinate ascent.

Dates et versions

hal-02362067 , version 1 (13-11-2019)

Identifiants

Citer

Ahmet Alacaoglu, Olivier Fercoq, Volkan Cevher. On the convergence of stochastic primal-dual hybrid gradient. SIAM Journal on Optimization, 2022, 32 (2), pp.1288-1318. ⟨10.1137/19M1296252⟩. ⟨hal-02362067⟩
32 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More