To the average eye, extended reality is starting to look bleak. The metaverse has bombed, the Apple Vision Pro has flopped, and Sony has all but abandoned the PSVR.
Sadly for Mark Zuckerberg, consumers rarely want to strap computers to their faces. But there is one place where business is booming: the military.
XR has diffused across the armed forces since 2021, when Microsoft signed a contract with the US Army worth up to $21.9bn (€19.6bn). Under the deal, the tech giant would develop training programmes for HoloLens-based headsets.
Despite a shaky start — literally, for the nauseated soldiers — the partnership continues to this day. But it might not last much longer: around 80 firms are now vying for the contract.
As the market has expanded, new use cases have emerged. You can now find XR in endless military applications, from combat rehearsals and battlefield tactics to vehicle training and helmet systems. But that’s just on the ground.
Arguably the fastest-growing military application for XR is pilot training. In this segment, there’s an undisputed European champion: Varjo.
From its headquarters in Finland, Varjo (pronounced “var-yo”) is building a thriving business in synthetic flight training. The company’s CEO, Timo Toikkanen, says a “paradigm shift” has begun.
The trigger was evolving needs for aircraft simulators. Advances in XR have paved a path towards compelling new systems.
One was recently delivered to Ukraine. It will support the country’s latest aerial weapons: F-16 fighter jets.
Flight preparations
After years of lobbying western allies, Ukraine finally received its first F-16 fighter jets in August. The shipment marked a milestone for the country’s air forces, which have hitherto relied on Soviet-era jets.
F-16s add a powerful upgrade to the fleet. But there’s a problem with the order: Ukraine doesn’t have enough pilots who can fly the planes.
Traditional solutions to this problem come with their own issues. Aircraft training is extremely expensive, seats are limited, and the courses often require long-distance travel. The conventional alternative is dome simulators, but these are also costly to develop. They’re also vast machines that require their own dedicated buildings.
XR can hurdle these barriers. The simulators are cheaper than domes and can operate in offices. They could also integrate systems from across the armed forces. But their biggest strength today is their speed.
“The training time of a fighter pilot is compressed by 30 to 50%,” Toikkanen says. “When every year costs millions, that’s a very significant change.”
Ukraine has an urgent need for fast and affordable training. F-16s could intercept enemy jets and establish areas of air superiority — but only with enough pilots to fly them. To train them, the country recently acquired its first fully-functional XR F-16 system.
Czech firm Dogfight Boss built the simulator, while Varjo supplied the XR tech.
After entering the cockpit, pilots can learn the controls, fine-tune their strategies, and fly virtual missions.
The simulator brings a welcome boost to Ukraine’s air forces. But for Varjo, the country is a tiny addition to a ballooning global market.
“The F-16 in Ukraine is one example of a much broader phenomena,” Toikkanen said. “And the phenomena is traditional means of training pilots being replaced by mixed reality technology.”
XR takes off
One of Varjo’s closest partners is Aechelon Technology. The American company creates geo-specific visualisations of the real world, which have been integrated with Varjo’s XR system. Together, the companies have crafted headsets for the US Air Force.
“We are one of the big players in the US. But Varjo is the player,” Javier Castellar, the co-founder of Aechelon, tells TNW.
Castellar estimates that Varjo has captured at least 95% of the XR flight training market. He calls the firm “the Tesla of Finland.”
This reputation in military aviation has blossomed rapidly. A few years ago, XR wasn’t technically capable of replacing air force simulators. Fast forward to today, and Varjo is in over 80 military synthetic training programs across NATO’s footprint. Orders for XR, Castellar says, now outstrip those for domes by at least eight to one.
The reason for this turnaround is a big leap in tech.
Varjo’s breakthrough headset was the XR-4 series. Released last year, the devices blend a 360-degree view of the synthetic environment with the cockpit interior. Castellar says the system “crossed a threshold of human vision.”
Inside the headset, foveated rendering tracks the pilot’s eyes and maximises the resolution where they’re looking. By using this technique, the XR-4 boosts the visual quality while cutting the computation needs. Dual 4K x 4K displays can then deliver photorealistic scenes at 90 frames per second.
To integrate the pilot’s surroundings, two 40-megapixel cameras align the visual focus with the pilot’s gaze. When their eyes shift from the digital surroundings to the physical cockpit, the pass-through system shifts the view from the virtual to the real. TNW got to test the tech last year and found the transition seamless.
As the uptake grows, new capabilities are emerging. “It’s not just a display system,” says Castellar. “It has major implications on defence.”
Flying higher
Modern aircraft are expensive to update. In domes, the costs are cut, but the changes still don’t come cheap. They can also involve laborious implementations.
XR promises a simpler solution. ”The architecture can be continuously adapted, because it becomes more of a software problem,” says Castellar.
The enhancements are potentially endless. On an F-16 simulator, you could add night vision goggles, new weapon systems, or the latest aircraft helmet. All these components can then enter mission rehearsals.
But that’s only for aircraft. Varjo expects XR to spread across armed forces and into complex military operations. The headsets will connect planes in the sky with ships at sea, and vehicles on the ground. Numerous simulators will join the same scenario. Every element will network into one environment.
“You can’t practise that in real life,” Toikkanen says. “The only practice is to simulate.”
The software also offers advanced data analytics. You can measure a pilot’s cognitive stress, reaction speeds, or gaze direction. Training programs can be tailored to their needs.
The insights could even shape real military equipment. “You can make pilots fly aircraft that don’t exist yet,” Castellar says. “You can experience the effects of the performance on the battlefield and make design decisions based on that.”
This affordable adaptability creates a recurrent business model. In the future, Castellar envisions a service that resembles smartphone upgrade programmes.
“I think it’s going to bring profits in a sustained way, which is unusual in this industry,” he says. “It’s normally a one-off purchase.”
As evidence of the benefits increases, Castellar expects the uptake to snowball. The defence market is typically slow-moving. But when new tech gains traction, order numbers often proliferate.
They can also lead to new applications. Endless military machines could follow the XR flightpath.
“These are no longer just training aids that are nice to have,” Castellar says. “They’re becoming absolutely fundamental for the defence posture of every country.
Get the TNW newsletter
Get the most important tech news in your inbox each week.