Prepared in 2016

Linearly convergent randomized iterative methods for computing the pseudoinverse
Robert M. Gower and Peter Richtárik
Preprint, December 2016

Randomized distributed mean estimation: accuracy vs communication
Jakub Konečný and Peter Richtárik
Preprint, November 2016

Federated learning: strategies for improving communication efficiency
Jakub Konečný, H. Brendan McMahan, Felix Yu, Peter Richtárik, Ananda Theertha Suresh and Dave Bacon
In NIPS Private Multi-Party Machine Learning Workshop, 2016
[poster]

Federated optimization: distributed machine learning for on-device intelligence
Jakub Konečný, H. Brendan McMahan, Daniel Ramage and Peter Richtárik
Preprint, October 2016

A new perspective on randomized gossip algorithms
Nicolas Loizou and Peter Richtárik
to appear in the 4th IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2016

AIDE: fast and communication efficient distributed optimization
Sashank J. Reddi, Jakub Konečný, Peter Richtárik, Barnabás Póczos, Alex Smola
Preprint, June 2016
[poster]

Coordinate descent face-off: primal or dual?
Dominik Csiba and Peter Richtárik
Preprint, May 2016

Optimization in high dimensions via accelerated, parallel and proximal coordinate descent
Olivier Fercoq and Peter Richtárik
SIAM Review 58(4), 739-771, 2016
SIAM SIGEST Award

Stochastic block BFGS: squeezing more curvature out of data
Robert M. Gower, Donald Goldfarb and Peter Richtárik
In Proceedings of The 33rd International Conference on Machine Learning, 1869-1878, 2016
[poster]

Importance sampling for minibatches
Dominik Csiba and Peter Richtárik
Preprint, February 2016

Randomized quasi-Newton updates are linearly convergent matrix inversion algorithms
Robert M. Gower and Peter Richtárik
Preprint, January 2016
[code: SIMI, RBFGS, AdaRBFGS, ...]


Prepared in 2015

Even faster accelerated coordinate descent using non-uniform sampling
Zeyuan Allen-Zhu, Zheng Qu, Peter Richtárik and Yang Yuan
In Proceedings of The 33rd International Conference on Machine Learning, 1110-1119, 2016
[code: NU_ACDM]

Stochastic dual ascent for solving linear systems
Robert M. Gower and Peter Richtárik
Preprint, December 2015
[code: SDA] YouTube

Distributed optimization with arbitrary local solvers
Chenxin Ma, Jakub Konečný, Martin Jaggi, Virginia Smith, Michael I Jordan, Peter Richtárik and Martin Takáč
Optimization Methods and Software, 1-36, 2017

Distributed mini-batch SDCA
Martin Takáč, Peter Richtárik and Nathan Srebro
Preprint, 2015

Randomized iterative methods for linear systems
Robert M. Gower and Peter Richtárik
SIAM Journal on Matrix Analysis and Applications 36(4), 1660-1690, 2015
2nd Most Downloaded Paper from the SIMAX website (Aug 2016)

[slides] [code: mSDCA]

Primal method for ERM with flexible mini-batching schemes and non-convex losses
Dominik Csiba and Peter Richtárik
Preprint, 2015
[code: dfSDCA]

Mini-batch semi-stochastic gradient descent in the proximal setting
Jakub Konečný, Jie Liu, Peter Richtárik and Martin Takáč
IEEE Journal of Selected Topics in Signal Processing 10(2), 242-255, 2016
[code: mS2GD]

On the complexity of parallel coordinate descent
Rachael Tappenden, Martin Takáč and Peter Richtárik
Preprint, 2015

Stochastic dual coordinate ascent with adaptive probabilities
Dominik Csiba, Zheng Qu and Peter Richtárik
In Proceedings of The 32nd International Conference on Machine Learning, 674-683, 2015
Best Contribution Award (2nd Place), Optimization and Big Data 2015
Implemented in Tensor Flow
[poster] [code: AdaSDCA and AdaSDCA+]

Adding vs. averaging in distributed primal-dual optimization
Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter Richtárik and Martin Takáč
In Proceedings of The 32nd International Conference on Machine Learning, 1973-1982, 2015
2015 MLconf Industry Impact Student Research Award link
CoCoA+ is now the default linear optimizer in Tensor Flow link
[poster] [code: CoCoA+]

SDNA: Stochastic dual Newton ascent for empirical risk minimization
Zheng Qu, Peter Richtárik, Martin Takáč and Olivier Fercoq
In Proceedings of The 33rd International Conference on Machine Learning, 1823-1832, 2016
[slides] [poster] [code: SDNA]


Prepared in 2014

Coordinate descent with arbitrary sampling II: expected separable overapproximation
Zheng Qu and Peter Richtárik
Optimization Methods and Software 31(5), 858-884, 2016

Coordinate descent with arbitrary sampling I: algorithms and complexity
Zheng Qu and Peter Richtárik
Optimization Methods and Software 31(5), 829-857, 2016
[code: ALPHA]

Semi-stochastic coordinate descent
Jakub Konečný, Zheng Qu and Peter Richtárik
To appear in Optimization Methods and Software
[code: S2CD]

Quartz: Randomized dual coordinate ascent with arbitrary sampling
Zheng Qu, Peter Richtárik and Tong Zhang
In Advances in Neural Information Processing Systems 28, 865-873, 2015
[slides] [code: QUARTZ] YouTube

mS2GD: Mini-batch semi-stochastic gradient descent in the proximal setting
Jakub Konečný, Jie Liu, Peter Richtárik and Martin Takáč
In NIPS Workshop on Optimization for Machine Learning, 2014
[poster] [code: mS2GD]

S2CD: Semi-stochastic coordinate descent
Jakub Konečný, Zheng Qu and Peter Richtárik
In NIPS Workshop on Optimization for Machine Learning, 2014
[poster] [code: S2CD]

Simple complexity analysis of simplified direct search
Jakub Konečný and Peter Richtárik
Preprint, 2014
[slides in Slovak] [code: SDS]

Distributed block coordinate descent for minimizing partially separable functions
Jakub Mareček, Peter Richtárik and Martin Takáč
Numerical Analysis and Optimization, Springer Proceedings in Math. and Statistics 134, 261-288, 2015

Fast distributed coordinate descent for minimizing non-strongly convex losses
Olivier Fercoq, Zheng Qu, Peter Richtárik and Martin Takáč
In 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2014
[poster] [code: Hydra^2]

On optimal solutions to planetesimal growth models
Duncan Forgan and Peter Richtárik
Preprint, 2014

Matrix completion under interval uncertainty
Jakub Mareček, Peter Richtárik and Martin Takáč
European Journal of Operational Research 256(1), 35-42, 2017
[code: MACO]


Prepared in 2013

Accelerated, Parallel and PROXimal coordinate descent
Olivier Fercoq and Peter Richtárik
SIAM Journal on Optimization 25(4), 1997-2023, 2015
17th IMA Leslie Fox Prize (Second Prize), 2015
2nd Most Downloaded Paper from the SIOPT website (Aug 2016)
[poster] [code: APPROX] YouTube

Semi-stochastic gradient descent methods
Jakub Konečný and Peter Richtárik
Preprint, 2013
[poster] [slides] [code: S2GD and S2GD+]

On optimal probabilities in stochastic coordinate descent methods
Peter Richtárik and Martin Takáč
Optimization Letters 10(6), 1233-1243, 2016
[poster] [code: NSync]

Distributed coordinate descent method for learning with big data
Peter Richtárik and Martin Takáč
Journal of Machine Learning Research 17(75), 1-25, 2016
[poster] [code: Hydra]

Smooth minimization of nonsmooth functions with parallel coordinate descent methods
Olivier Fercoq and Peter Richtárik
Preprint, 2013
[code: SPCDM]

Separable approximations and decomposition methods for the augmented Lagrangian
Rachael Tappenden, Peter Richtárik and Burak Buke
Optimization Methods and Software 30(3), 643-668, 2015

Inexact coordinate descent: complexity and preconditioning
Rachael Tappenden, Peter Richtárik and Jacek Gondzio
Journal of Optimization Theory and Applications 170(1), 144-176, 2016
[poster] [code: ICD]

TOP-SPIN: TOPic discovery via Sparse Principal component INterference
Martin Takáč, Selin Damla Ahipasaoglu, Ngai-Man Cheung and Peter Richtárik
Preprint, 2013
[poster] [code: TOP-SPIN]

Mini-batch primal and dual methods for SVMs
Martin Takáč, Avleen Bijral, Peter Richtárik and Nathan Srebro
In Proceedings of the 30th International Conference on Machine Learning, 2013 
[poster] [code: minibatch SDCA and minibatch Pegasos]


Prepared in 2012

Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
Peter Richtárik, Martin Takáč and Selin Damla Ahipasaoglu
Preprint, 2012
[code: 24am]

Optimal diagnostic tests for sporadic Creutzfeldt-Jakob disease based on SVM classification of RT-QuIC data
William Hulme, Peter Richtárik, Lynne McGuire and Alison Green
Preprint, 2012

Parallel coordinate descent methods for big data optimization
Peter Richtárik and Martin Takáč
Mathematical Programming 156(1), 433-484, 2016
16th IMA Leslie Fox Prize (2nd Prize), 2013
[slides] [code: PCDM, AC/DC] YouTube


Prepared in 2011

Efficient serial and parallel coordinate descent methods for huge-scale truss topology design
Peter Richtárik and Martin Takáč
Operations Research Proceedings 2011, 27-32, Springer-Verlag, 2012
[poster]

Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
Peter Richtárik and Martin Takáč
Mathematical Programming 144(2), 1-38, 2014
Best Student Paper (runner-up), INFORMS Computing Society, 2012
[slides]

Efficiency of randomized coordinate descent methods on minimization problems with a composite objective function
Peter Richtárik and Martin Takáč
Proceedings of Signal Processing with Adaptive Sparse Structured Representations, 2011

Finding sparse approximations to extreme eigenvectors: generalized power method for sparse PCA and extensions
Peter Richtárik
Proceedings of Signal Processing with Adaptive Sparse Structured Representations, 2011


Prepared in 2010 or earlier

Approximate level method for nonsmooth convex minimization
Peter Richtárik
Journal of Optimization Theory and Applications 152(2), 334–350, 2012

Generalized power method for sparse principal component analysis
Michel Journée, Yurii Nesterov, Peter Richtárik and Rodolphe Sepulchre
Journal of Machine Learning Research 11, 517–553, 2010
[slides] [poster] [code: GPower]

Improved algorithms for convex minimization in relative scale
Peter Richtárik
SIAM Journal on Optimization 21(3), 1141–1167, 2011
[slides]

Simultaneously solving seven optimization problems in relative scale
Peter Richtárik
Technical Report, 2009

Some algorithms for large-scale convex and linear minimization in relative scale
Peter Richtárik
PhD Dissertation, School of Operations Research and Information Engineering, Cornell University, 2007