Reinforcement Learning is a branch of machine learning where AI (agent) aims to make the most optimal decision in a given situation (state) through interactions.

Terminology

  • Agent: AI
  • Environment: What an agent interacts with
  • State: What agent can see. This is often a strict subset of the environment. (e.g: Environment: whole map, state: what an agent can currently see in first-person). $s$
  • Action: How an agent interacts with the environment to change the state. (e.g: going forward, turning around, etc). $a$
  • Reward: How favourable an action is to achieve the goal. $r$
  • Policy: Strategy to choose an action given a state. (e.g if a robot encounters an obstacle in front, a reasonable policy would instruct the robot to avoid it). Often written as $\pi$
  • State-Value (or Value): Favourability of a given state. How good it is to end up in the state. Written as $V^\pi(s)$
  • Action-Value (or Q-Value): Favourability of a given action in a given state. How good it is to choose that action in a given state. Written as $Q^\pi(s, a)$
  • Return: Cumulative reward from timestamp t and thereafter.
  • Advantage: How good an action is compared to other possible actions. $A(s,a) = Q(s,a) - V(s)$