Reinforcement Learning 101
Reinforcement Learning is a branch of machine learning where AI (agent) aims to make the most optimal decision in a given situation (state) through interactions.
Terminology
- Agent: AI
- Environment: What an agent interacts with
- State: What agent can see. This is often a strict subset of the environment. (e.g: Environment: whole map, state: what an agent can currently see in first-person). $s$
- Action: How an agent interacts with the environment to change the state. (e.g: going forward, turning around, etc). $a$
- Reward: How favourable an action is to achieve the goal. $r$
- Policy: Strategy to choose an action given a state. (e.g if a robot encounters an obstacle in front, a reasonable policy would instruct the robot to avoid it). Often written as $\pi$
- State-Value (or Value): Favourability of a given state. How good it is to end up in the state. Written as $V^\pi(s)$
- Action-Value (or Q-Value): Favourability of a given action in a given state. How good it is to choose that action in a given state. Written as $Q^\pi(s, a)$
- Return: Cumulative reward from timestamp t and thereafter.
- Advantage: How good an action is compared to other possible actions. $A(s,a) = Q(s,a) - V(s)$