Reinforcement Learning Algorithms: Analysis and Applications

Boris Belousov is a Ph.D. student at Technische Universität Darmstadt, Germany, advised by Prof. Jan Peters. He received his M.Sc. degree from the University of Erlangen-Nuremberg, Germany, in 2016, supported by a DAAD scholarship for academic excellence. Boris is now working toward combining optimal control and information theory with applications to robotics and reinforcement learning.

Hany Abdulsamad is a Ph.D. student at the TU Darmstadt, Germany. He graduated with a Master's degree in Automation and Control from the faculty of Electrical Engineering and Information Technology at the TU Darmstadt. His research interests range from optimal control and trajectory optimization to reinforcement learning and robotics. Hany's current research focuses on learning hierarchical structures for system identification and control.

This book reviews research developments in diverse areas of reinforcement learning such as model-free actor-critic methods, model-based learning and control, information geometry of policy searches, reward design, and exploration in biology and the behavioral sciences. Special emphasis is placed on advanced ideas, algorithms, methods, and applications.

The contributed papers gathered here grew out of a lecture course on reinforcement learning held by Prof. Jan Peters in the winter semester 2018/2019 at Technische Universität Darmstadt.

The book is intended for reinforcement learning students and researchers with a firm grasp of linear algebra, statistics, and optimization. Nevertheless, all key concepts are introduced in each chapter, making the content self-contained and accessible to a broader audience.

Prediction Error and Actor-Critic Hypotheses in the Brain.-  Reviewing on-policy / o -policy critic learning in the context of Temporal Di erences and Residual Learning.- Reward Function Design in Reinforcement Learning.- Exploration Methods In Sparse Reward Environments.- A Survey on Constraining Policy Updates Using the KL Divergence.- Fisher Information Approximations in Policy Gradient Methods.- Benchmarking the Natural gradient in Policy Gradient Methods and Evolution Strategies.- Information-Loss-Bounded Policy Optimization.- Persistent Homology for Dimensionality Reduction.- Model-free Deep Reinforcement Learning - Algorithms and Applications.- Actor vs Critic.- Bring Color to Deep Q-Networks.- Distributed Methods for Reinforcement Learning.- Model-Based Reinforcement Learning.- Challenges of Model Predictive Control in a Black Box Environment.- Control as Inference?