Hooking up brains to machines could be the ‘next big thing’ for gaming

It may sound like science fiction, but brain-machine interfaces (BMI) will eventually come to our devices. It’s just a matter of time. The technology has been actively researched since 1969, before consumer computers even existed, when a researcher showed that monkeys could amplify their brain signals to control a needle on a dial.

Today, researchers are making strides to bring brain-machine interfaces closer to reality. Stanford researchers successfully created an interface for brain-controlled typing where users simply imagine their own hand movements to move an on-screen cursor, helping paralyzed users communicate.

Startup CTRL-Labs has figured out a way to make the brain-machine interface possible without the need for an intrusive implant thanks to an electromyography (EMG) wristband and machine learning.

For mobile, a brain-machine interface could be a pivotal moment in computing. The keyboard has been around since the genesis of the computer, and its replacement will come in the form of a brain-to-machine technology.

Touch screens were seen as a major step forward, allowing users to interact directly with elements on the screen instead of using a mouse, but brain-machine interfaces are entirely different and have the ability to fundamentally change how we interact with our devices. And just as with new technologies like virtual reality, augmented reality, and motion tracking, gaming will be the industry that pushes brain-machine interfaces into the mainstream.

Mind games

Beyond the obvious use of brain-machine interfaces assisting the disabled, the technology could also change how we use devices in the future, and one of the most local implementations is in gaming. Like the Nintendo Gameboy that ushered in the era of handheld devices (superseding the Palm Pilot), and powerful graphics cards once used for gaming now turned into tools for design and Bitcoin mining, gaming will push forward an era of brain-controlled input devices in the near future.

Brain-machine interfaces offer a solution to the current problems with mobile gaming, namely that our hands and fingers block most of the action happening on screen. Additionally, touch screen controls just aren’t good at providing us with feedback. Physical controls give us palpable signals to know when we’re pressing harder or if we’re providing the intended input.

This just can’t be replicated with smartphones, even with new technologies like Apple’s 3D Touch and Taptic Engine. While Fortnite may be huge on mobile, there are still plenty of gamers who prefer the feedback and input methods of console and PC.

Shifting perspectives to VR, we’re still constrained by clunky controllers instead of using our hands naturally. Today’s VR controllers are a necessary evil until more natural ways of communicating with machines are achieved, like with CTRL-Labs’ brain-machine interface or Leap Motion’s hand tracking tech.

Hand tracking is nice, but it still creates an extra layer of interpretation (a camera must recognize and interpret our gestures) rather than giving our brains a direct input path.

By getting rid of controllers, gaming could feel a lot more natural and approachable, and as a result, BMI technology will be more widely accepted and adopted. “We’re working toward the ability for users to be able to walk up, put the band on, not have to do any training, and be able to roll right way,” said Mike Astolfi, head of interactive experiences at CTRL-Labs, speaking with Engadget.

We’re already seeing companies experiment with brain-to-brain interfaces, which allows participants to collaborate on a game of 20 questions. Another startup called Neurable is working on a VR game that lets players interact with virtual objects telepathically via its accessory for the HTC Vive. “We wanted to create a system that would require basically no training,” said Neurable CEO Rames Alcaide speaking with IEEE Spectrum.

Once brain-machine interfaces take off, VR experiences will be much more immersive, as players can simply interact with virtual environments like they would in real life without the need for a controller acting as a middleman. Additionally, gaming experiences, whether in VR, AR, or other means, can finally be provided with real-time feedback to adapt experiences to us.

For example, imagine a horror game that detects when we get increasingly nervous and ramps up the horror factor. On the flipside, you could have games and apps that help users relax via measurable feedback via heart rate and muscle tension.

By lowering the learning curve for various technology platforms, brain-machine interfaces could bring VR into the mainstream as touch controls did for the smartphone. Simultaneously, BMI input devices will be brought into the mainstream by VR.

Challenges moving forward

While brain-machine interfaces are an inevitability, they face a number of issues that must be solved before it can brought to market as a computing input method, let alone as a gaming peripheral.

The biggest problems brain-machine interfaces must solve for gaming are accuracy and latency. Gaming requires high accuracy and low latency in order for players to feel in control and be competitive. Any hint of lag will frustrate players.

his is why professional esports athletes still prefer wired peripherals even though wireless keyboards, mice, and controllers have become the norm for a majority of consumers.

Of course, with sufficient advancement, brain-machine interfaces could theoretically offer the least amount of latency out of controlling options; brain-to-software with no physical interface between is the shortest possible path.

Furthermore, brain-machine interfaces are only one part of the equation for changing the gaming paradigm. Games will likely rely on multiple input devices and data sources like heart rate monitors and eye tracking to make truly immersive experiences a possibility.

Trying to miniaturize all of this tech into an unobtrusive headset or other peripheral will be a massive challenge for the near future. But whatever the growing pains are, the struggle will be worth it for the new experiences and accessibility brain-machine interfaces bring.