On Reinforcement Learning
Learning by trial-and-error
Reinforcement learning is an exciting area of work that has seen significant advances in recent years, and will likely see major advances in the coming years. In particular, this field is particularly well-positioned to benefit from the expected windfall in compute power. In fact, it was one of the pre-eminent researchers in RL, Richard Sutton, that wrote the now well-known essay "The Bitter Lesson". What makes RL so powerful is that it is a general method for learning how to act in an unknown environment, without necessarily having to construct a detailed model of that environment. While RL is starting to get more attention now, the fundamentals of RL go farther back. The foundations of RL - Markov Decision Processes, Dynamic Programming, Bellman's equation, and of course probability theory are important pre-requisites for RL. This lens assumes the reader understands probability theory. If you are not, I suggest taking the time to learn it, and learn it well. This lens will build on that knowledge.
Tap to Preview

Foundations

  • The Bitter Lesson

  • Dynamic Programming

  • Markov Decision Process (MDP) Tutorial

  • Reinforcement Learning: An Introduction

  • Intro to Reinforcement Learning — Monte Carlo to Policy Gradient

    • Exhaustive Tabular Methods

  • Principles of Autonomy and Decision Making: MDPs and Value Iteration

  • Principles of Autonomy and Decision Making: Policy Iteration

    • Monte Carlo and Temporal Difference Methods

  • Monte Carlo Learning

  • 4.6 BlackJack with First visit MC.ipynb

    • Function Approximation

  • RL Course by David Silver - Lecture 6: Value Function Approximation

  • Reinforcement Learning Part 7: Value Function Approximation

    • Deep RL

  • Temporal Difference Learning and TD-Gammon

  • TD-Gammon: Reinforcement Learning Algorithm for Backgammon

  • Playing Atari with Deep Reinforcement Learning

    • Policy Gradient

  • RL Course by David Silver - Lecture 7: Policy Gradient Methods

  • Policy Gradient Methods for Reinforcement Learning with Function Approximation

  • REINFORCE Algorithm: Reinforcement Learning from Scratch in PyTorch

  • CartPole Balance with REINFORCE Policy Gradient

  • Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

  • Trust Region Policy Optimization

  • Proximal Policy Optimization Algorithms