Lukas Balles

I’m a doctoral student in machine learning at the International Max Planck Research School for Intelligent Systems, hosted by the Max Planck Institute for Intelligent Systems and the University of Tuebingen. I am supervised by Prof. Philipp Hennig.



I am primarily working on optimization methods for machine learning with a focus on deep learning. Optimization algorithms are the number crunching workhorse of contemporary machine learning! Intriguingly, numerical optimizers themselves can be seen as compact little “learning machines”: they make decisions (where to evaluate next, how many and which data points to use) based on observations (stochastic evaluations of function values and gradients). My goal is to design smarter optimizers!

One focus area of my research is the use of (estimates of) the stochastic gradient (co-)variance to make the optimizer aware of the stochasticity of the evaluations it receives. I believe that this can help improve various aspects of stochastic optimization algorithms. For example, gradient variance estimates can be used to adaptively choose batch sizes when performing stochastic gradient descent (Balles et al., 2017). Variance estimates can also be used to manipulate the update direction itself by “damping” directions with low signal-to-noise ratio (Balles and Hennig, 2018). This is related to preconditioning with the so-called empirical Fisher matrix (Kunstner, Balles, and Hennig, 2019).

Check out my Google Scholar page for a list of publications. For some projects, code can be found on my GitHub page.

Short Bio

Prior to joining the Max Planck Institute as a Ph.D. student, I studied Mathematics (B.Sc.) and Scientific Computing (M.Sc.) at Heidelberg University and spent some time as a visiting student at Tsinghua University in Beijing. I did my Master’s thesis research project at (what is now) the Bosch Center for Artificial Intelligence. During my PhD, I interned in the Google Brain team in Montreal and at Google AI in Zurich.

Get in Touch

E-mail: first-name dot last-name at tuebingen dot mpg dot de.

Twitter: @lukas_balles