Researchers teach Hearthstone bot to dominate Legend rank players

A team of researchers from the University of Warsaw recently developed an AI bot to play Blizzard’s Hearthstone. This isn’t the first machine built to play the game, but it may very well set the benchmark for all that come after it.

Hearthstone is a collectible card game, like Magic the Gathering or Yu-Gi-Oh. Its rules and format make it a great candidate for exploration by machine learning developers. Like poker, it requires players to work with both shared and limited information. The player that outperforms the other player tends to win, despite the fact that luck also factors in.

According to the researchers, they’ve created a bot with the potential to be competitive in Hearthstone matches — similar to how DeepMind’s AlphaGo was developed to defeat humans. The conclusion of their white paper, published yesterday, states:

The bot is able to win, with an impressive consistency, 100% games against the random player. It is also capable of winning games against Legend rank players, which alone can be regarded as very promising. The human players reported that in many situations they felt the bot played really well.

Online collectible card games typically come with built in “AI,” in the form of a scripted bot. But the challenge of created an unscripted bot capable of winning against highly skilled human players (without cheating) isn’t the same at all. A scripted bot only works in a protected environment – like a single player mode or tutorial.

To function in an unstructured environment, bots are built using decision tree algorithms to make choices on behalf of the AI or they’re made with deep learning networks. The Warsaw team’s bot has both. It essentially combines a popular heuristic search algorithm called the Monte Carlo Tree Search with a supervised deep learning network. The result is a bot that should be competitive among both other bots and humans.

Despite its popularity among AI researchers, there doesn’t yet exist a recognized benchmark for machine learning models built to play Hearthstone. According to the Warsaw team:

We can conclude that the resulting agent is likely to be among the strongest Hearthstone bots at the moment. Although Hearthstone has become a testbed for AI, there has not been yet proposed any universal benchmarking methods, so it is difficult to assess the strength other than by human observation, self-play between various versions of the agent or a random player.

The team submitted their bot for competition at the 2018 Hearthstone AI Competition at the Conference on Computational Intelligence and Games this week.