Part III · Era 5 — Deep Learning Revolution (2012–2016)

draft

The turning point. Deep networks trained on GPU over lots of data surpass everything that came before — first in vision, then in games. Modern AI begins here.

Narrative

In 2012, AlexNet wins the ImageNet challenge by a huge margin, proving that deep convolutional networks + GPUs are a leap, not an increment. Computer vision is rewritten in months; soon come advances in language (word2vec), generation (GANs) and translation (attention). In 2016, AlphaGo defeats a world Go champion — a milestone that was believed to be decades away. Deep learning goes from a bet to a consensus.

Milestones

  • 2012 — AlexNet · Krizhevsky, Sutskever, Hinton · ImageNet error plummets;

    start of the revolution.

  • 2013 — word2vec · Mikolov (Google) · embeddings that capture word

    semantics.

  • 2013/2015 — DQN plays Atari · DeepMind · deep RL learns from scratch

    through pixels.

  • 2014 — GANs · Ian Goodfellow · realistic generation through an adversarial

    game.

  • 2014 — Seq2seq + Attention · Sutskever; Bahdanau · neural translation; seed

    of the Transformer.

  • 2015 — ResNet · He et al. (Microsoft) · residual connections enable

    ultra-deep networks.

  • 2015 — TensorFlow · Google · framework that popularizes deep learning.
  • 2016 — AlphaGo beats Lee Sedol · DeepMind · historic milestone in

    perfect-information games.

People

Ilya Sutskever (1986–) — coauthor of AlexNet and seq2seq. Later cofounder and chief scientist of OpenAI. 🖼️ Photo F-III.13.

Ian Goodfellow (1985–) — inventor of GANs (2014). Opened up adversarial data generation.

Demis Hassabis (1976–) — cofounder of DeepMind. AlphaGo, AlphaFold; Nobel Prize in Chemistry 2024. 🖼️ Photo F-III.14.

Alex Krizhevsky — first author of AlexNet. Implemented the GPU training that made the leap possible.

🎨 Figure F-III.15 — The cat that changed everything. Brief: iconic illustration of a CNN processing the image of a cat in layers (edges → textures → parts → object), with a GPU board pulsing alongside; in the background, AlphaGo's Go board with "move 37". Celebrates the moment when vision and games fell.

Connection to the Compendium: it is the explosion of the connectionist paradigm (../02-types-of-ai/02-connectionist.kmd) and of deep RL (../02-types-of-ai/05-reinforcement.kmd); it enshrines GPU + data + training as the core of the life cycle (Part I).