4TUNE

Presentation

4TUNE is an associate team within CWI-Inria International Lab.

The long-term goal of 4TUNE is to push adaptive machine learning to the next level. We aim to develop refined methods, going beyond traditional worst-case analysis, for exploiting structure in the learning problem at hand. We will develop new theory and design sophisticated algorithms for the core tasks of statistical learning and individual sequence prediction. We are especially interested in understanding the connections between these tasks and developing unified methods for both. We will also investigate adaptivity to non-standard patterns encountered in embedded learning tasks, in particular in iterative equilibrium computations.

Ongoing projects

Koolen, Gaillard and Taylor have started a project on bandit convex optimisation, where the objective is to adapt to the unknown function and noise structure. The first steps will be to work out information-theoretic lower bounds and make them operational, which requires optimisation over function classes (this is one of the strong points of the INRIA Paris team). We will further draw on saddle-point methods developed at CWI to obtain provable algorithms.

Koolen, Gaillard, and Taylor have started another project on projection-free learning, where the objective is to fill the gap between projection-based and linear optimization-based methods in online learning. For doing that, we rely on the idea of « mimicking » projection-based method using projection-free ones. The first steps rely on the idea of analyzing performances of projection-based methods when the potentially expansive projection steps are roughly approximated, and on analyzing the capability of the other methods for providing those estimates.

Publications

The list below contains papers related to the 4TUNE project. Those works fit 4TUNE's themes on adaptivity (and its underlying cost), as well as the development of flexible algorithms with with provable guarantees going beyond pure worst-case analysis.

• Lipschitz and comparator-norm adaptivity in online learning. Z. Mhammedi, W. M. Koolen. In Proceedings of the 33rd Annual Conference on Learning Theory (COLT), 2020.
• Structure adaptive algorithms for stochastic bandits. R. Degenne, H. Shao, W. M. Koolen. In Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.
• Non-stationary Online Regression. A. Raj, P. Gaillard, C. Saad. Technical report, 2020.
• Efficient improper learning for online logistic regression. J. Rémi, P. Gaillard, A. Rudi. In Proceedings of the 33rd Annual Conference on Learning Theory (COLT), 2020.
• Complexity guarantees for Polyak Steps with momentum. M. Barré, A. Taylor, A. d’Aspremont. Proceedings of the 33rd Annual Conference on Learning Theory (COLT), 2020.
• Convergence of Constrained Anderson Acceleration. M. Barré, A. Taylor, A. d’Aspremont. Technical report, 2020.
• Principled Analyses and Design of First-Order Methods with Inexact Proximal Operators. M. Barré, A. Taylor, F. Bach. Technical report, 2020.

Contact us

Please visit contact information on researchers' webpages.