Rainbow, The Colorful Evolution of Deep Q-Networks

This article has been published on Towards Data Science, read it here!

Intro

In 2013, the introduction of Deep Q-Networks (DQN) by Mnih et al. marked the first breakthrough in Deep Reinforcement Learning, surpassing expert human players in three Atari games. Over the years, several variants of DQN were published, each improving on specific weaknesses of the original algorithm.

In 2017, Hessel et al. made the best out of the DQN palette by combining 6 of its powerful variants, crafting what could be called the DQN Megazord: Rainbow.

In this article, we’ll break down the individual components that make up Rainbow, while reviewing their JAX implementations in the Stoix library.