The Sequence Knowledge #804: The Dreamer Trilogy: Inside Some of the Most Influential Papers in AI World Models
3 Papers that changed how we think about world models.
Today we will Discuss:
An overview of model-free RL inspired world models and the Dreamer efforts.
A review of the three Dreamer papers.
💡 AI Concept of the Day: Model-Based RL as the Inspiration for World Models
The transition from 2D arcade games to 3D environments was the crucible that forged the modern World Model.
In the mid-2010s, “Model-Free” Reinforcement Learning (RL) was enjoying a golden age. Agents like DQN were crushing human scores in Atari games.1 These agents were reactive; they mapped pixels directly to actions. In a 2D side-scroller, if you see a pit, you jump. The correlation between the visual input and the correct action is direct and immediate.
However, when researchers tried to apply these same methods to 3D environments—like DeepMind Lab or Minecraft—progress stalled.