Reinforcement Learning · 15 problems · 2 project ideas

Problem sheet

Start with the essentials5 problems

Discounted Return

returns · discounting · rewards

Gridworld Transition Function

environments · mdp · gridworld

The Environment Interface: reset and step

environments · gymnasium-api · episodes

Gymnasium Environments

gymnasium · environments · spaces

Epsilon-Greedy Action Selection

exploration · epsilon-greedy · policies

Connect the concepts5 problems

Incremental Value Estimates

bandits · incremental-mean · step-size

UCB1 Bandit

bandits · ucb · exploration

Iterative Policy Evaluation

dynamic-programming · bellman-equation · policy-evaluation

Value Iteration

dynamic-programming · value-iteration · optimal-policy

Policy Iteration

policy-iteration · dynamic-programming · policy-improvement

Stretch your understanding5 problems

First-Visit Monte Carlo Prediction

monte-carlo · prediction · returns

TD(0) Prediction

temporal-difference · td-learning · bootstrapping

Tabular Q-Learning

q-learning · control · off-policy

Q-Learning on FrozenLake

gymnasium · q-learning · frozenlake

SARSA

sarsa · on-policy · control

Ungraded briefs to build on your own2 project ideas

Grid Game Agent

Project idea · reinforcement-learning · gymnasium · q-learning

Bandit-Based Recommender

Project idea · bandits · recommendation · exploration