Open course

I have been teaching a course on reinforcement learning for five years. Along with my teaching, I developed a textbook and an open course. The development process was very time-consuming. I sincerely hope they are helpful to those who are interested in reinforcement learning!

Lecture Videos

You can check out our YouTube channel. You can also click the following links to be directed to specific lecture videos.

• Overview of Reinforcement Learning in 30 Minutes

• L1: Basic Concepts (P1-State, action, policy, ...)

• L1: Basic Concepts (P2-Reward,return, Markov decision process)

• L2: Bellman Equation (P1-Motivating examples)

• L2: Bellman Equation (P2-State value)

• L2: Bellman Equation (P3-Bellman equation-Derivation)

• L2: Bellman Equation (P4-Matrix-vector form and solution)

• L2: Bellman Equation (P5-Action value)

• L3: Bellman Optimality Equation (P1-Motivating example)

• L3: Bellman Optimality Equation (P2-Optimal policy)

• L3: Bellman Optimality Equation (P3-More on BOE)

• L3: Bellman Optimality Equation (P4-Interesting properties)

• L4: Value Iteration and Policy Iteration (P1-Value iteration)

• L4: Value Iteration and Policy Iteration (P2-Policy iteration)

• L4: Value Iteration and Policy Iteration (P3-Truncated policy iteration)

• L5: Monte Carlo Learning (P1-Motivating examples)

• L5: Monte Carlo Learning (P2-MC Basic-introduction)

• L5: Monte Carlo Learning (P3-MC Basic-examples)

• L5: Monte Carlo Learning (P4-MC Exploring Starts)

• L5: Monte Carlo Learning (P5-MC Epsilon-Greedy-introduction)

• L5: Monte Carlo Learning (P6-MC Epsilon-Greedy-examples)

• L6: Stochastic Approximation and SGD (P1-Motivating example) 

• L6: Stochastic Approximation and SGD (P2-RM algorithm: introduction) 

• L6: Stochastic Approximation and SGD (P3-RM algorithm: convergence) 

• L6: Stochastic Approximation and SGD (P4-SGD algorithm: introduction) 

• L6: Stochastic Approximation and SGD (P5-SGD algorithm: examples) 

• L6: Stochastic Approximation and SGD (P6-SGD algorithm: properties) 

• L6: Stochastic Approximation and SGD (P7-SGD algorithm: comparison) 

• L7: Temporal-Difference Learning (P1-Motivating example)

• L7: Temporal-Difference Learning (P2-TD algorithm: introduction)

• L7: Temporal-Difference Learning (P3-TD algorithm: convergence)

• L7: Temporal-Difference Learning (P4-Sarsa)

• L7: Temporal-Difference Learning (P5-Expected Sarsa & n-step Sarsa)

• L7: Temporal-Difference Learning (P6-Q-learning: introduction)

• L7: Temporal-Difference Learning (P7-Q-learning: pseudo code)

• L7: Temporal-Difference Learning (P8-Unified viewpoint and summary)

• L8: Value Function Approximation (P1-Motivating example – curve fitting)

• L8: Value Function Approximation (P2-Objective function) 

• L8: Value Function Approximation (P3-Optimization algorithm)

• L8: Value Function Approximation (P4-illustrative examples and analysis)

• L8: Value Function Approximation (P5-Sarsa and Q-learning)

• L8: Value Function Approximation (P6-DQN–basic idea)

• L8: Value Function Approximation (P7-DQN – experience replay)

• L8: Value Function Approximation (P8-DQN – implementation and example)

• L9: Policy Gradient Methods (P1-Basic idea)

• L9: Policy Gradient Methods (P2-Metric 1–Average value)

• L9: Policy Gradient Methods (P3-Metric 2–Average reward)

• L9: Policy Gradient Methods (P4-Gradients of the metrics)

• L9: Policy Gradient Methods (P5-Gradient-based algorithms & REINFORCE)

• L10: Actor-Critic Methods (P1-The simplest Actor-Critic)

• L10: Actor-Critic Methods (P2-Advantage Actor-Critic)

• L10: Actor-Critic Methods (P3-Importance sampling & off-policy Actor-Critic)

• L10: Actor-Critic Methods (P4-Deterministic Actor-Critic)

• L10: Actor-Critic Methods (P5-Summary and goodbye!)

Lecture Slides

The slides for the above lecture videos can be found here: https://github.com/MathFoundationRL/Book-Mathematical-Foundation-of-Reinforcement-Learning/tree/main/Lecture%20slides 

Textbook

The GitHub homepage of the textbook is here: https://github.com/MathFoundationRL/Book-Mathematical-Foundation-of-Reinforcement-Learning

So far, the textbook has received 7,000+ stars on GitHub. Unlike many other books, this book provides a mathematical but friendly introduction to reinforcement learning. Its content structure is also novel. You are welcome to check out!

Lecture Videos (in Chinese)

For Chinese lecture videos, you can check our Bilibili channel or our YouTube channel

So far, the lecture videos have received 1,300,000+ views over the Internet and received very good feedback! You are welcome to check out!