How AI will change storytelling and become the next great art form

It seems self-defeating for the former head of Oculus Stories Studios and current founder and executive producer of VR startup Fable to declare AI, not virtual reality, the next big art form. Especially considering that, aside from him, nobody knows what that means.

TNW talked to Fable founderEdward Saatchi about his company’s pivot away from focusing on just making stories in VR. He told us that the future of entertainment was “virtual beings,” non-living characters powered by AI that interact with humans.

If you’re thinking of Alexa or Cortana right now, you’re not really wrong, but that’s not what Saatchi’s talking about. According to him, we’ll never be friends with them because they’re servants. “People don’t want to be around someone who always wants to do whatever you want to do. Friendships require friction,” he told TNW.

And, if you think about it, it’s true. Even if you love how useful your virtual assistant is, it’s not your friend. Your friends don’t hunch down on your counter motionlessly waiting for you to call their name. “Hey John, do friend stuff.”

What Saatchi is talking about is more like plucking a character out of your favorite book and allowing you to interact with them as they are — not as you tell them to be.

In fact, if your favorite book is Neil Gaiman’s “The Wolves in the Walls” you’re in luck: Fable’s making the protagonist Lucy a virtual being as part of a three-chapter VR adaptation of the book.

In the latest chapter, “Whispers In The Night,” viewers can interact with a Lucy who responds and reacts to them as if you’re really in the room with her. This chapter just debuted at Sundance, but Fable says the work will be available in the Oculus Store once all three chapters are completed.

The idea here is that, as a virtual being, Lucy can share her memories with you, discuss them with you, and interact with you in real-time. She’ll even be able to remember what you’ve said to and done with her the next time she sees you.

In one way, you’ll be able to visit her in her own world. She’s a virtual person in a virtual land: you’ll use a headset to see her and your voice and hands to interact with her.

But, in another way, Lucy doesn’t live inside your Oculus Rift headset or Gaiman’s book. She exists for the same reason Peter Pan’s Tinkerbell does: because you want her to. The characters that inspire, delight, terrify, and move us live forever.

Saatchi wants people to have personal access to the characters they care about, but he has no interest in attempting to make virtual beings that try to pass as sentient or alive. To hear him tell it, pursuing AI designed to pass a modern version of the Turing test is pointless.

He’s far more pragmatic and grounded than his “AI is the next great art form” statements would make him seem. He believes AI will become a conduit for art in the same way that books, radios, and televisions did.

If you can imagine a character with memories, mannerisms, a distinct voice and appearance, and a compelling raison d’être, you’re halfway to being a best-selling fiction writer. But what can you do with that character?

Let’s take a classic one: Captain Ahab from Herman Melville’s “Moby Dick.” How do we put him in your iPhone where, through the magic of augmented reality, we can see him sitting on the couch in our own living room? Maybe you want to smoke a pipe with him and ask why he wanted to be a fisherman to begin with, or what it was like walking aboard the Pequod for the first time.

And what if we want to put on a VR headset and join him aboard the ship as he tells that story? How does a developer seamlessly send Ahab from our phone to our headset, keeping his memory of our interaction (including my name and how much he’s already told me about himself) intact?

Moreover, how do we make Ahab’s chest heave when he gets excited, his fist clench when we remind him of something he doesn’t like, or get him to pace up and down our hallway while stopping to look out a window or stare us down?

The answer is artificial intelligence. Saatchi told TNW:

We’ll have a team where immersive storytellers work on the story aspect and other teams where machine learning experts build the characters. So, you might have an AI team working on the legs, and one working on the arms and hands, and another working on the facial expressions. These might even be different companies.

The big picture points to a paradigm where virtual beings can exist in any digital medium because the AI powering them can adapt to the situation. Because of this, it will feel like the character is aware of you and engaging you, not because it’s been animated to do so, but because it’s reacting to you as an individual.

But isn’t that what Amazon, Google, Microsoft, and Apple are already working on with virtual assistants that are supposed to eventually be able to carry a real conversation with us? Not really.

No amount of money or machine learning know-how can replace the artistic nuance of a Neil Gaiman, or a Stephen King, or a Piers Anthony. Big tech isn’t working on the same thing that Fable and the other companies involved in creating virtual beings are.

The difference between a servant AI or chat-bot AI, and a virtual being based on a character, is in the storytelling. If you download a virtual Batman, it shouldn’t be happy to change the volume on your connected home speakers. That’s a waste of Batman’s time and “he” should tell you as much.

Alexa, on the other hand, had better turn on the damn lights when it’s told or else it begins to lose value. If we tell it to set an alarm, we expect the alarm to go off when it’s supposed to. Otherwise, it’s a gadget that doesn’t work right.

But storytellers are beholden to canon. And the characters we love, be they Hobbits or Jedi Knights, should always stay true to themselves. Otherwise you wind up in a situation like the one Activision and Neversoft found themselves in when they made the late Kurt Cobain a playable character in Guitar Hero 5, thus making it possible to use his likeness to perform any song in the game.

We should never, under any circumstances, see Kurt Cobain singing a pop song. It’s jarring and, according to Cobain’s family and former bandmates, offensive.

As this idea relates to fictional characters, it’s about canon. Saatchi puts it:

It doesn’t make sense, in my opinion, for Lucy to change the lights in your home from red to blue or something like that.

But the problem with building characters that people can interact with isn’t about whether or not the developers will do them justice. Those kinds of concerns have been around since the invention of the sequel.

As a fan, when you put your beloved characters in someone else’s hands, – even the original creator’s – you always risk ending up with “Indiana Jones and the Kingdom of the Crystal Skull,” when you were hoping for “Indiana Jones and the Temple of Doom.” AI won’t solve this.

What AI can do is make it as comfortable and intuitive as possible for us to interact with virtual characters. If the character I’m interacting with doesn’t turn its head when I move, make eye-contact, and walk like I expect a person to walk, I lose immersion immediately.

Storytellers currently use tricks to get around this because it’s simply not possible to record an animation or motion-capture for every possible interaction.

In Robert Rodriguez’ “The Limit,” a VR movie experience in which you’re the main character, he gets around the problem by giving you control over nothing but where you’re looking.

So when Michelle Rodriguez’ character says “take my hand,” you see an actor’s arm reach up from your virtual body whether you move or not. It works, and it’s a clever way around the fact that she’s not really there waiting for you to reach up and take her hand, but it’s a useless technique for unscripted interactions.

Eventually, even the most clever of storytellers will run out of ways to make it seem like you’re really interacting with something.

What Fable and Saatchi are attempting to accomplish isn’t about making a ‘Lucy AI’ based on Gaiman’s work. Lucy ‘exists’ whether she’s controlled by AI, Gaiman’s word processor, or your imagination. It’s about giving us immersive access to those characters on a personal level, and allowing us to interact with them as though they’re old friends  — something our favorite characters almost always are.

Edward Saatchi, Fable, and several other companies recently announced the Virtual Beings Conference. It will take place in San Francisco this summer. For more information visit the official website here.