Reinforcement Learning — Winter 2024

Summarize brief history of reinforcement learning.

Illustrate the general idea of reinforcement learning using Tic-Tac-Toe example.

Describe the Elements of Reinforcement Learning in detail.

Define and formulate probability density function (PDF)

Explain concepts of random variables in probability with suitable example.

Discuss the concept of joint, and conditional probability with their equations.

Define Probability Mass Function (PMF)? Explain with suitable example.

Illustrate Markov Reward Process (MRP) with suitable example.

What are the roles of Optimal Value Functions in Markov decision process (MDP)?

Formulate Policy Evaluation to compute the state-value function in dynamic programming.

What is The Markov Property? List the criteria to identify the Markov Property.

Solve the Bellman equation for v∗ for the simple Gridworld problem.

Explain Policy Iteration in detail.

Give overview of Monte Carlo (MC) methods for model free reinforcement learning.

Illustrate with example, On policy and off policy learning.

Write an algorithm for the first-visit MC method for estimating v . 𝜋

Explain how to apply Importance Sampling in off-policy technique.

Take down value iteration algorithm.

Write an algorithm for the Every-visit MC method for estimating v . 𝜋

Define Temporal-Difference Learning (TD). Give the overview of Overview TD (0).

What are the advantages of TD Prediction Methods?

Explain Q-Learning for an off-policy TD control algorithm.

What it means by the term “Eligibility Traces”? What are two ways to view eligibility traces?

Differentiate State-action-reward-state-action (SARSA) and Q-learning

Describe N-step TD prediction technique in detail.