Overview: Here’s how Skype will work on the Xbox One

Back in May, Microsoft first announced Skype would be bundled with the Xbox One. Since then, we’ve seen glimpses of Skype working on the console, but nothing major as the company was still working hard to nail down as much functionality as possible.

In Stockholm, Sweden, I had the opportunity to talk to Todd Roshak, Lead Program Manager of Skype for Xbox at Microsoft. He revealed planning started in July 2012, with the first code being written three months later in October 2012. That’s about 16 months of work for those of you too lazy to do the math, and while discussing and getting a tour of the new app, I got the feeling that the Skype team was only just getting started.

The majority of the team working on Skype for Xbox was either hired (many engineers came straight from Sony) or came from Microsoft (such as the Windows 8 team), not from Skype. Almost all of them who didn’t already work in London, England, had to relocate for the job.

How it works

The first thing the Skype team for Xbox got up and running was the chat client, which lets you input on the virtual keyboard with the console’s controller. This lets you ping to see if your contact is there, use emoticons, and so on. Next, audio and video calls were added, followed by all the other features like being able to add accounts.

The former required making sure that Skype’s Silk audio codec could work in conjunction with Xbox’s hardware. The team had to swap out Skype’s existing audio and video engines for the hardware accelerated engines on the console.

In fact, the Xbox One is the first device where Skype doesn’t use its own engines, according to Mo Ladha, Microsoft’s Product and Program Manager for Skype and Lync Video. This allows the Xbox One’s game chat and party chat to be based on the Silk codec, using a common technology for all types of calls on the console.

Navigating the Skype client for Xbox One works just as you’d expect: you can use the controller or your voice. There are three sections to start from, just like other Skype apps: Recent, Favorites, and People.

Video calls have two features worth underlining: auto zoom and auto pan. Because the Kinect comes with the Xbox One, Skype can track skeletons to keep track of everyone in the room. If someone leaves the frame, Skype will focus on who is left and zoom in on them. Roshak says this functionality was pulled off as part of a “last-minute effort.”

Here is the behind-the-scenes look:

And here are the two features in action:

This works particularly well in group video calling, which not only tracks everyone in the frame, but also lets you bring individuals to fullscreen if they have something important to say or show. Otherwise, you will see them side-by-side.

Unfortunately, the feature is limited to four people right now. The good news is that Ladha told us more than 90 percent of group calls are four or less people. Naturally that number will likely jump once Skype for Xbox One, which is much better suited for large groups than laptops and desktops, increases the maximum.

Yet the Skype client is more than just an app on the console. It is ubiquitous through the system itself, part of the Xbox One’s fast switching feature. This is the first Skype client that is more than just Skype on a device: Skype for Xbox One has system-wide integration.

This means you can simply say “Xbox, go to Skype” to fire up the app. You can also say “Xbox, call Emil” to call me and “Xbox, hang up” to stop talking to me.

We asked Microsoft for a full list of global voice commands (you can use these from anywhere on the Xbox One), and here is what they came back with:

• “Call *person’s name*” (from your favorites list).
• “Skype *person’s name*” (from your favorites list).
• “Go to Skype”.
• “Hang up”.
• “Answer”.
• “Answer without video”.
• “Ignore”.

The first two points are worth underlining: you have to first add a contact to the Favorites section if you want to call them by using their name (“Call Emil”). This is required so that Xbox can have a finite list of users to listen for. Unfortunately, the Favorites limit is currently set to 10.

Just like any other app on the Xbox One, there are additional commands that can be used in the actual Skype app. These depend on what screen you’re currently looking at, as the Xbox One can listen for you to say most text that is visible on your TV.

Not all parts of Skype are speakable (like adding a contact or dictating in a chat session), but Roshak says these features are on their way. Chat, however, does work with the SmartGlass keyboard, and the app actually lets you control Skype to an extent, not just the Xbox One.

While Skype is addressable at all times, meaning you can get calls while playing games or call others while gaming, you can also set yourself as invisible so nobody bothers you when you’re about to frag your friend. There are three settings for notifications: Show All Notifications (IMs included), Show Call Notifications, and Don’t Show Notifications.

Speaking of receiving calls when you’re in another app: you can keep talking even when you’re using say, Internet Explorer. Pulling this off was “one of the single hardest challenges we had over the project,” Roshak said.

So, what happens when you’re in game chat and you get a Skype call? Essentially, the Xbox One offers you a call waiting option, so you can temporarily pick up another call. When you hang up, you’ll go right back to game chat.

Last but not least, you can rate the call you just had, just like on any other Skype release. The team wanted this to be on the Xbox One so it can get direct feedback from day one. “It’s incredibly important to keep our quality up and improving it all the time” emphasized Karlheinz Wurm, General Manager in Product and Test for Skype and Lync Real Time Media.

Microsoft’s goal with Skype for Xbox One was a very big one: instead of building a client for face-to-face interactions, expand the experience to sharing experiences together. In other words, events such as birthdays, family dinners, and so on were all on the table.

To do that, the Skype team had to take advantage of the hardware (the Kinect’s 1080p camera, array microphone, echo cancellation, beam forming, and so on) without impacting performance of the console’s main use case: gaming. In other words, Skype needs to be able to dynamically scale itself from 1080p video to audio only, without any stuttering in game framerate.

Performance was very tough to pull off. Xbox is the first to do bring a full-fledged Skype video client into the living room, including group chat. Furthermore, this was all done on very limited resources (the team had to push very hard on metrics to get it working under their set budgets, Roshak underlined). The team had to look at how to limit both foreground and background tasks, and a lot of these improvements will be carried over to other devices, Ladha told us.

Since Skype comes with the Xbox One, the app’s text will be localized for each of the 13 launch countries. Skype global and local voice commands will be available at launch in the US, the UK, France, Germany, Canada, Australia, Brazil, Mexico, Italy, and Spain.

Bonus feature: this one isn’t documented in the Xbox One FAQ, but I had the opportunity to see it anyway. Skype’s screensharing function was added after some quick work, mainly because Roshak’s kids were taking a distance learning course at home.

The engineers decided to keep it enabled so users can discover it by chance. It was an easy one to add simply because screensharing uses the h.264 format, and so does the Xbox One, so the Skype team merely needed to add the screen as another video stream.

What doesn’t work, yet

While the Xbox One definitely features a fully functional Skype client, there is a lot missing. Roshak specifically highlighted two features that aren’t available at launch, but when pushed, he revealed quite a few more.

First up, you can’t favorite groups. This means you can’t say “Xbox, call *group-name*.” Roshak says this is because currently, some Skype groups have names while others are simply an amalgamation of their members’ names. As a result, Skype will need to make some changes for this functionality to be added, but Roshak promised it’s on the way.

Next, the People section currently has no way to filter who is online and who is offline. This was a big annoyance amongst those who got to test Skype for Xbox One early, and Roshak specifically said this would be coming “pretty soon.”

You also can’t promote any of your Xbox Live friends to become Skype contacts. Microsoft understandably wants to be very careful about sharing your real information that’s available on Skype to gamers, most of whom are anonymous and are only known by their gamer tags. One day, the company will likely offer such an optional feature, but it first wants to make sure it can be as transparent as possible.

While you can send links over Skype and open them in Internet Explorer on the Xbox, sending files currently doesn’t work (you’ll get a generic error saying the functionality isn’t supported on this Skype client). The Xbox team thinks receiving most files isn’t useful, but it is looking at handling pictures and videos because those can of course be viewed on the console.

Yet none of these compare to the lack of being able to snap Skype on the right-hand side, especially given that Microsoft touted the feature extensively. In other words, you can’t play games while being on a Skype video call.

When you tell a gamer that they will be able to use Skype on the Xbox One, this is probably the first scenario they imagine. The only way you can currently perform a Skype call while gaming is keeping the call in the background, an audio-only connection as you can see above. Video simply is not an in-game option available at launch.

What’s next

In addition to the above section, which should give you a decent idea as to what the Skype for Xbox team is currently working on, there are still a few things the engineers are staying tight-lipped about. We do know, however, that the first update to Skype for Xbox will be released in the next few days.

It will let gamers use the Kinect camera to take a picture for their profile, but don’t expect any big feature additions beyond that. Roshak confirmed with me that snap won’t be ready, and that the first update is primarily a bug-fix release.

In other words, no Xbox One gamer in 2013 (including the critical holiday season) will be able to use Skype as a gamer would expect: while playing games. They’ll have to settle for Skype audio-only chats while gaming, and all the usual Skype functionality mentioned above.

Then again, being able to order a pizza from your console is pretty awesome. There’s a lot of room for Skype to grow in the living room.

See also – Xbox One review: A multimedia extravaganza that also plays games and Skype is considering automatically removing typing sounds from calls

Disclosure: Microsoft paid for The Next Web’s trip to Skype’s Stockholm offices but has no influence at all over anything we publish.

Top Image Credit: Justin Sullivan/Getty Images