Part II · Gallery — Reinforcement AI

draft

Intelligence that learns by trial and error, maximizing reward by interacting with an environment. Inspired by the psychology of conditioning; it is the paradigm of behavior and control.

🎮 Deep RL (value) Reinforcement + connectionist · Games/control · Neural network + value function (Q)

  • What it is: learns which action has the highest expected value in each state.
  • Examples: DQN playing Atari (DeepMind, 2013/2015).
  • Algorithmic basis: Q-learning, experience replay, target networks.
  • Peak: 2013–2017.
  • Capabilities / modes: reactive control; learns from scratch through experience.
  • → Part IV: chapter planned.

🏆 Policy RL / AlphaZero Reinforcement + connectionist · Board games/planning · Policy + MCTS

  • What it is: combines a policy/value network with tree search for perfect

    mastery of games.

  • Examples: AlphaGo (2016), AlphaZero, MuZero.
  • Algorithmic basis: policy gradient, Monte Carlo Tree Search,

    self-play.

  • Peak: 2016–2020.
  • Capabilities / modes: strategic/intellectual; surpasses humans at games.
  • → Part IV: chapter planned.

🎚️ Continuous RL / Control Reinforcement + connectionist · Robotics/continuous control · Actor-critic

  • What it is: learns policies in continuous action spaces.
  • Examples: PPO, SAC, DDPG; robot locomotion, industrial control.
  • Algorithmic basis: actor-critic, trust region, entropy regularization.
  • Peak: 2017–present.
  • Capabilities / modes: bodily-kinesthetic (motor control).
  • → Part IV: chapter planned.

💬 RLHF / RLAIF Reinforcement + connectionist · Language alignment · Reward model + PPO/DPO

  • What it is: aligns LLMs by using human (or AI) feedback as reward.
  • Examples: InstructGPT, ChatGPT, Claude (Constitutional AI/RLAIF).
  • Algorithmic basis: reward model, PPO, DPO (direct).
  • Peak: 2022–present.
  • Capabilities / modes: alignment of behavior and style.
  • → Part IV: chapter planned.

Salient sciences and mathematics: psychology (conditioning), control theory, game theory, optimization, stochastic processes (MDPs), economics (utility). Strength: learns behavior without labels; weakness: sample-inefficient and sensitive to reward design.