Watch: Russian researchers can control drone swarms like Jedi

Did you know Neural is taking the stage on Sept 30 and Oct 1? Together with an amazing line-up of experts, we will explore the future of AI during TNW Conference 2021. Secure your ticket now!

A team of AI researchers in Russia have managed to come up with one of the coolest gesture-control systems for drones that we’ve seen yet. What’s even more impressive is that it works with multiple machines simultaneously.

Controlling swarms of drones in real-time is the dream. And, when you pre-program them with advanced software and run them in unobstructed areas it’s pretty easy to accomplish.

But if you want to, say, control a swarm of drones inside an office building, in real time, things get a bit trickier.

Enter the Skolkovo Institute of Science and Technology’s AI team:

Up front: It’s no small feat creating a gesture-control system for drones. First off, flying objects have their own physics to compensate for and keeping them from slamming into each other and surrounding objects is a monumental task.

Then you’ve got to figure out how to keep them from colliding with things in real-time – an undertaking made exponentially more complex if both operator and drones are entering an unexplored area.

Per the researchers’ pre-print paper:

We propose a novel human-swarm interaction system, allowing the user to directly control a swarm of drones in a complex environment through trajectory drawing with a hand gesture interface based on the DNN-based gesture recognition.

Background: The researchers developed “Drone Paint,” a novel control system that relies on gestures to control drones. It’s called Drone Paint because, well, it’s pretty amazing at light-painting.

Light-painting is a technique that involves taking a long exposure photograph of light in motion to produce an image that appears to have objects painted on it with light.

Quick take: It might be difficult to visualize usefulness from the light-painting demo, but when you start to imagine AI/human hybrid control methods things get interesting.

A soldier, for example, could control drone swarms in real-time in the battle zone by simply waving them towards targets (buildings with dangerous corners and shadows, for example). They wouldn’t have to glance away from the tactical arena in order to push a button on a smartphone or speak into a microphone.

There are also applications for directing drone swarms through disaster areas or on search missions in crowded areas such as city centers or sports stadiums.

This is a very cool use of computer vision and, with further development, we could see drone technology pushing its way to the forefront of useful AI systems.