Sony’s new AI agent achieves superhuman Gran Turismo Sport scores

One of the best things about computers is that they can learn just as much from a simulation as they can from so-called ‘real world’ experiences. That means, given the proper simulator, we can teach AI to drive cars without ever putting a single human in danger.

Just about every AI company trains their driverless vehicle algorithms using simulations. Until now, the simulators themselves weren’t all that interesting. They’re mostly just physics engines designed to be interpreted by a neural network. But Sony just unveiled the most popular autonomous driving simulator ever: Gran Turismo Sport.

In case you’re not a gamer: this isn’t advanced software designed to train AIs, it’s a game. And not just any game but the latest in one of the most beloved racing simulation series in history

Researchers from the University of Zurich and Sony AI Zurich recently published a pre-print paper showcasing the development of an autonomous agent designed to beat the best human players at the game.

Per the team:

Among racing games, Gran Turismo Sport (GTS) is known as a highly realistic driving simulation, modelling phenomena, such as the influence of tires’ temperature and a car’s current fuel level on traction. Therefore, similarly to real-world racing, the optimal trajectory (i.e., the trajectory leading to the fastest lap time) for a car in GTS depends not only on the geometry and properties of the track, but also on various (a priori unknown) physical characteristics and states of the car. Due to its similarity to real driving, and the relatively low price of training in GTS compared to training with actual race cars, GTS is also used to cast drivers for racing teams.

In other words: It’s a legitimate simulation that’s used by real-world race teams to help determine real, expert-level drivers’ abilities. That’s pretty high praise for a video game.

The researchers had a pretty tall order to fill. While AI systems regularly outperform humans in games such as chess and Go, standard computer-controlled racers tend to fair poorly against expert human players.

The researchers write:

To our knowledge, the built-in non-player characters (NPC) included in modern car racing games are unable to compete with human expert players in fair comparisons. For example, the currently built-in NPC in Gran Turismo Sport (GTS) loses a total of 11 seconds compared with the fastest human driver and is slower than 83% of all humans in one of our reference settings.

Other racing games apparently close the gap to human experts by granting an unfair advantage to the NPC, for example by increasing the engine power of the NPC’s car; this, however, leads to frustration among human players who feel cheated.

Rather than cheat or tweak the rules, the team turned to a facet of AI called deep reinforcement learning. This involved training the AI to recognize the road ahead and react in a more human-like fashion.

According to an article by Tech Xplore writer Ingrid Fadella, Yunlung Song, a co-author on the team’s research paper, said:

Different from classical state estimation, trajectory planning and optimal control methods, our approach does not rely on human intervention, human expert data, or explicit path planning. We found that it could generate trajectories that are qualitatively similar to those chosen by the best human players, while outperforming the best known human lap times in all three of our reference settings, including two different cars on two different tracks.

To the best of our knowledge this is the first time an autonomous car AI has beaten human experts in Gran Turismo Sport. And while there currently exists no artificial intelligence system capable of level five autonomy (able to drive a vehicle with no external aids or human-assistance), if you absolutely must ride in a vehicle controlled by an AI: may as well pick the one trained in a video game about pushing the physical limits of speed and control.

You can read the whole paper here.

So you’re interested in AI? Then join our online event, TNW2020, where you’ll hear how artificial intelligence is transforming industries and businesses.