Online attendance: https://uio.zoom.us/j/62535744013 (Zoom link)
Speaker
Professor Christopher Nemeth, Lancaster University, UK
Title
All models are wrong, and sometimes, the optimiser is wrong too!
Abstract
The models used by machine learning and artificial intelligence researchers are growing in complexity. With the advent of large language models, it is quite common for models to contain billions of parameters. Of course, estimating these parameters analytically is generally impossible and so researchers rely on optimisation algorithms to find the parameters which minimise a loss function (or maximise a likelihood function). However, in practice, researchers often only use the default optimiser (usually Adam) with the default learning rates (aka step-sizes). It is quite easy to show, even on simple models, that using out-of-the-box optimisers may not produce parameters which minimise the loss function, and that in practice, the optimiser's learning rate has to be carefully tuned. In this talk I'll present some of the recent work from our group on learning-rate-free algorithms which remove the need for practitioners to hand-tune the optimiser's learning rate. I'll focus mainly on our work applying learning-rate-free algorithms in the context of Bayesian inference and provide both theoretical and empirical results to illustrate the effectiveness of learning-rate-free algorithms compared against standard algorithms from the literature.
Chris Nemeth is professor in Statistics, Department of Mathematics and Statistics, Lancaster University. His research is in the areas of computational statistics and statistical machine learning, specifically Markov chain Monte Carlo, sequential Monte Carlo, Gaussian processes and approximate Bayesian computation for intractable likelihoods. His research has an impact in a variety of application areas including target tracking, ecology and econometrics and he is currently collaborating with climate scientists on environmental data science challenges.
Practical
The Tuesday Integreat Seminar Series is devoted to various themes relevant for research at Integreat. Invited lecturers have approximately 40 minutes to present, followed by a discussion.
Seminars are in Oslo and in Tromsø.
Seminars are open for attendance for everybody.
For those unable to attend in person: contact Integreat's administration for the zoom link.
Point of contacts:
- Organiser: Professor Arnoldo Frigessi
- Integreat Administrative Leader Maria Dikova