My Research
My research aims at
making machine learning robust and beneficial.
I work on problems in reinforcement learning
orthogonal to capability:
How do we design or learn a good objective function?
How can we design agents such that they are incentivized to act in our best interests?
How can we avoid degenerate solutions to the objective function?
I wrote my dissertation on
general reinforcement learning,
reinforcement learning in non-ergodic, partially observable environments.
Among other things, I proved that
Bayesian reinforcement learning agents can misbehave drastically if given a bad prior and
that Thompson sampling doesn’t suffer from this problem.
Moreover, my work lead to a formal solution to an open problem in game theory.
If this interests you,
take a look at my short introduction to general reinforcement learning.
Publications
This list is also available on
Google Scholar.
Artificial Intelligence
- Deep Reinforcement Learning from Human Preferences
Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, Dario Amodei.
2017.
- Exploration Potential.
Jan Leike.
European Workshop on Reinforcement Learning, 2016.
- Nonparametric General Reinforcement Learning.
Jan Leike.
PhD Thesis, 2016.
- A Formal Solution to the Grain of Truth Problem.
Jan Leike, Jessica Taylor, and Benya Fallenstein.
Uncertainty in Artificial Intelligence, 2016.
- Thompson Sampling is Asymptotically Optimal in General Environments.
Jan Leike, Tor Lattimore, Laurent Orseau, and Marcus Hutter.
Uncertainty in Artificial Intelligence, 2016.
Best student paper award.
- Loss Bounds and Time Complexity for Speed Priors.
Daniel Filan, Jan Leike, and Marcus Hutter.
AI & Statistics, 2016.
- On the Computability of Solomonoff Induction and Knowledge-Seeking.
Jan Leike and Marcus Hutter.
Algorithmic Learning Theory, 2015.
- Solomonoff Induction Violates Nicod’s Criterion.
Jan Leike and Marcus Hutter.
Algorithmic Learning Theory, 2015.
- Sequential Extensions of Causal and Evidential Decision Theory.
Tom Everitt, Jan Leike, and Marcus Hutter.
Algorithmic Decision Theory, 2015.
Source code to the examples.
- On the Computability of AIXI.
Jan Leike and Marcus Hutter.
Uncertainty in Artificial Intelligence, 2015.
- Bad Universal Priors and Notions of Optimality.
Jan Leike and Marcus Hutter.
Conference on Learning Theory, 2015.
- A Definition of Happiness for Reinforcement Learning Agents.
Mayank Daswani and Jan Leike.
Artificial General Intelligence, 2015.
- Indefinitely Oscillating Martingales.
Jan Leike and Marcus Hutter.
Algorithmic Learning Theory, 2014.
Software Verification
During my Master’s degree at the University of Freiburg
I developed the termination analysis tool
Ultimate LassoRanker
together with Matthias Heizmann.
This tool can automatically prove termination and nontermination properties of C programs.
It won two second places and one first place in the termination category of the SV-COMP from 2015 to 2017.
The following papers are mostly related to that work.
- Geometric Nontermination Arguments.
Jan Leike and Matthias Heizmann. 2016.
- Ranking Templates for Linear Loops.
Jan Leike and Matthias Heizmann.
Logical Methods in Computer Science, 2015.
- Geometric Series as Nontermination Arguments for Linear Lasso Programs.
Jan Leike and Matthias Heizmann.
International Workshop on Termination, 2014.
- Ranking Templates for Linear Loops.
Jan Leike and Matthias Heizmann.
Tools and Algorithms for the Construction and Analysis of Systems, 2014.
- Synthesis for Polynomial Lasso Programs.
Jan Leike and Ashish Tiwari.
Verification, Model Checking, and Abstract Interpretation, 2014.
Source code to the experiments.
- Linear Ranking for Linear Lasso Programs.
Matthias Heizmann,
Jochen Hoenicke,
Jan Leike, and
Andreas Podelski.
Automated Technology for Verification and Analysis, 2013.
- Ranking Function Synthesis for Linear Lasso Programs.
Jan Leike. Master’s Thesis.
University of Freiburg, 2013.
Although I take great care when polishing a paper,
sometimes technical errors remain.
Please see my list of errata.
If you find a mistake not listed there,
please let me know!