The new Xbox One Kinect tracks your heart rate, happiness, hands and hollers

Today at an event in Redmond, Microsoft revealed the new Xbox One console. In addition, it provided backstage tours with looks at new components like the freshly updated Kinect sensor.

This new Kinect is superior to the old one on just about every metric. Not only is it more compact, it’s also far more sensitive, with a 1080p RGB camera and an active infrared system that allows for precise tracking even in low-to-no light situations.

I was able to test the device out in a room with a bunch of other press types and the demonstrations were suitably impressive. First, we were shown how improved the depth graph of the Kinect is. This allows for precise placement of people in a room, as well as much better tracking of extremities, like fingers. The demo, which displayed our figures on the screen looking like cast pewter puppets, clearly showed fingers waving and had no trouble defining detailed objects like my badge.

• 
• 
• 
• 
• 
• 
• 
• 
• 

The field of view is 60% better than the last generation, and allows for six ‘skeletons’ to fit in the frame at once. it allows for the sensor to be used in smaller rooms as well, because people can get closer. The minimum object threshold is 2.5x better, allowing the use of fingers and hands more accurately.

The new Kinect has an active IR system for the first time, allowing the Xbox to identify you using the new ‘time of flight’ sensor. The new sensor captures more data, more accurate data across a larger field of view than the older version. The ‘time of flight’ measurement is the time it takes light to get to an object and back. This helps the Kinect correct for latency and produce more accurate results.

The new Kinect is also so accurate that it can pinpoint not just joints but also joint rotations. This allows it to see which direction you’re bending an arm, for instance. Going a step further, it can now track muscle tension based on body models. This means that it can interpret the amount of force being applied to both your body and say, a surface like the floor.

This also allows Kinect to track the speed and violence of movements like punches and kicks. The audio capabilities of the new Kinect were also demonstrated, showing how it could filter out a crush of game and room noise to pick out voice commands.

In a mind blowing demo, the Kinect then switched to a mode in which it monitored the heart rate of a person standing in front of it using the color cameras to measure how flush the skin was and the infrared cameras to track blood flow underneath the skin. This could ostensibly allow a developer to determine whether a user was scared, or even lying, and could also have health monitoring implications.

Then things started to get super freaky. A demo was run that showed the faces of people standing in front of the sensor. The Kinect was able to not only detect which controllers they were holding (player 1, player 2 etc), but also exactly who they were and whether they were happy, sad or neutral. This was done using imagery of the face to see whether they were smiling or frowning. It was pretty wild.

Though the demo was short, it was undoubtedly impressive. This new Kinect is vastly more sensitive than the previous model, which sometimes took a lot of arm flailing to boost into gear. We’ll have to see how it actually works with games once Microsoft gets around to demonstrating them.