Nuance’s Project Wintermute: A virtual cloud assistant that follows you across ecosystem boundaries

A personal assistant on every device. With Apple’s Siri and Google Now, that’s already becoming a reality for many smartphone and tablet users.

And it doesn’t seem far-fetched to think that Siri’s full capabilities will eventually come to the Mac and Apple TV. And Google Now to the various and sundry Android devices running Google’s version of the OS. But what about people with an Android phone and a Mac? Or a Windows PC and an iPhone? This cross-platform support will unequivocally never happen with Apple, and seems unlikely as Google moves to secure its mobile verticals.

That’s where Nuance’s Project Wintermute comes in.

Wintermute is an internal project at the company that’s attempting to create a cloud-based personal assistant that will follow you around, regardless of platform.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

“We want to be completely platform agnostic,” says Nuance VP and GM of Handset and Television Matt Revis tells me. “Nuance is Switzerland.”

And, in some ways, he’s right. The company’s notoriously ballsy CEO Paul Ricci was so confident in Nuance’s position as the best option for voice recognition and processing that he was rumored to refuse an acquisition offer by Apple founder Steve Jobs. Nuance would not be folded into Apple and disappear as a component of Siri’s voice capabilities. Instead, it would become the product that a wide array of partner companies turn to when they want ‘the best’ voice processing.

This position has allowed Nuance to make partnerships like a recently announced one with handset manufacturer ZTE, as well as its integration with Intel’s new ‘perceptive computing’ efforts (Nuance got a shout-out from the Intel press conference stage). It has also struck a deal with the Chrysler group to power its in-car text services and voice searching.

The Dragon TV product is a voice control system that Nuance is shipping in set-top boxes, cable boxes and Smart TVs. It allows, no surprise here, voice control of your television via casually delivered commands.

This cross-platform strategy is what Nuance is hoping to leverage to make Wintermute the assistant that crosses the boundaries between types of devices, brands and underlying operating systems. Apps for Android and iOS, embedded software on smart TVs, pre-installed software on Intel computers. Expanding to the car, other computing systems and whatever other platforms there are that support Nuance’s tech is likely in the cards as well.

I was given a demonstration of Project Wintermute in its early stages at this year’s CES. The demonstration came in two parts. An Android phone running Nuance’s assistant app was asked to play a song by Bob Dylan, which it began to do once the command had been recognized. The CES network issues were in full effect, however, when the command was delivered to a computer to ‘continue playing that song I was listening to’, and the command wasn’t recognized.

A second demo, however, worked flawlessly. After asking the phone to display the score of the Notre Dame game (and getting a response quickly, displayed visually much in the way that Siri does), we turned to a Dragon TV-enabled television.

“Put the game on,” the TV was asked. Note that there was no mention of which game, any team names or schedules at all. The TV instantly began playing the Notre Dame game, unprompted.

The information, of course, had been passed from the phone to Wintermute’s cloud. Then, when the assistant was invoked on another device, it used the information that it had gathered from the last query to infer that we wanted to watch, not just any game, but the game we had asked it the score of. It was quite impressive, frankly, and the implications were immediately evident.

As companies like Apple, Google, Microsoft and Amazon continue to do their best to lock us into their vertical silos of products, ecosystem baggage and features, we continue to lose our ability to choose. And yes, it may seem silly that we’re talking about the ability to pick our desired virtual assistant of choice, they’re still coming into being at a rapid clip. With Wintermute, you can move across devices and platforms with impunity, followed by your assistant, floating over your shoulder, dutifully noting your preferences and making sure it’s giving you what it thinks that you want.

If Wintermute can deliver, then it can continue to gather information that helps it to understand you better. And it can do that on an iPhone, in your car or on your TV. It doesn’t care where it is, as long as it’s interacting with you. It’s a bold concept, one that subsumes the traditional platform-centric nature of these personal assistants.

Revis noted to me that five years ago, voice control was all but unusable. At the current rate, where we’ll be in another five is hard to imagine.

Will we all have virtual personal assistants that are locked in their magic ecosystem bottles? Or ones that float along with us from device to living room and beyond, watching, remembering and serving? Nuance is betting on the latter with Wintermute.