Senior Research Scientist

DeepMind

Research Associate

Future of Humanity Institute

University of Oxford

Research Advisor

Machine Intelligence Research Institute

My research aims at making machine learning robust and beneficial. I work on problems in reinforcement learning orthogonal to capability: How do we design or learn a good objective function? How can we design agents such that they are incentivized to act in our best interests? How can we avoid degenerate solutions to the objective function?

I wrote my dissertation on general reinforcement learning, reinforcement learning in non-ergodic, partially observable environments. Among other things, I proved that Bayesian reinforcement learning agents can misbehave drastically if given a bad prior and that Thompson sampling doesn’t suffer from this problem. Moreover, my work lead to a formal solution to an open problem in game theory. If this interests you, take a look at my short introduction to general reinforcement learning.

This list is also available on Google Scholar.

**Deep Reinforcement Learning from Human Preferences**

Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, Dario Amodei. 2017.**Exploration Potential**.

Jan Leike. European Workshop on Reinforcement Learning, 2016.**Nonparametric General Reinforcement Learning**.

Jan Leike. PhD Thesis, 2016.**A Formal Solution to the Grain of Truth Problem**.

Jan Leike, Jessica Taylor, and Benya Fallenstein. Uncertainty in Artificial Intelligence, 2016.**Thompson Sampling is Asymptotically Optimal in General Environments**.

Jan Leike, Tor Lattimore, Laurent Orseau, and Marcus Hutter. Uncertainty in Artificial Intelligence, 2016.**Best student paper award**.**Loss Bounds and Time Complexity for Speed Priors**.

Daniel Filan, Jan Leike, and Marcus Hutter. AI & Statistics, 2016.**On the Computability of Solomonoff Induction and Knowledge-Seeking**.

Jan Leike and Marcus Hutter. Algorithmic Learning Theory, 2015.**Solomonoff Induction Violates Nicod’s Criterion**.

Jan Leike and Marcus Hutter. Algorithmic Learning Theory, 2015.**Sequential Extensions of Causal and Evidential Decision Theory**.

Tom Everitt, Jan Leike, and Marcus Hutter. Algorithmic Decision Theory, 2015. Source code to the examples.**On the Computability of AIXI**.

Jan Leike and Marcus Hutter. Uncertainty in Artificial Intelligence, 2015.**Bad Universal Priors and Notions of Optimality**.

Jan Leike and Marcus Hutter. Conference on Learning Theory, 2015.**A Definition of Happiness for Reinforcement Learning Agents**.

Mayank Daswani and Jan Leike. Artificial General Intelligence, 2015.**Indefinitely Oscillating Martingales**.

Jan Leike and Marcus Hutter. Algorithmic Learning Theory, 2014.

During my Master’s degree at the University of Freiburg I developed the termination analysis tool Ultimate LassoRanker together with Matthias Heizmann. This tool can automatically prove termination and nontermination properties of C programs. It won two second places and one first place in the termination category of the SV-COMP from 2015 to 2017. The following papers are mostly related to that work.

**Geometric Nontermination Arguments**.

Jan Leike and Matthias Heizmann. 2016.**Ranking Templates for Linear Loops**.

Jan Leike and Matthias Heizmann. Logical Methods in Computer Science, 2015.**Geometric Series as Nontermination Arguments for Linear Lasso Programs**.

Jan Leike and Matthias Heizmann. International Workshop on Termination, 2014.**Ranking Templates for Linear Loops**.

Jan Leike and Matthias Heizmann. Tools and Algorithms for the Construction and Analysis of Systems, 2014.**Synthesis for Polynomial Lasso Programs**.

Jan Leike and Ashish Tiwari. Verification, Model Checking, and Abstract Interpretation, 2014. Source code to the experiments.**Linear Ranking for Linear Lasso Programs**.

Matthias Heizmann, Jochen Hoenicke, Jan Leike, and Andreas Podelski. Automated Technology for Verification and Analysis, 2013.**Ranking Function Synthesis for Linear Lasso Programs**.

Jan Leike. Master’s Thesis. University of Freiburg, 2013.

Although I take great care when polishing a paper, sometimes technical errors remain. Please see my list of errata. If you find a mistake not listed there, please let me know!