What the latency between facial expression and mouseclick?

KinesicMouse forum
Post Reply
JoyfulRogue
Posts: 10
Joined: Fri May 22, 2015 12:07 pm

What the latency between facial expression and mouseclick?

Post by JoyfulRogue »

One of the biggest issues I've struggled with in hands-free gaming is the latency between the input and the mouseclick.

When I'm using voice commands, there is about a half second lag between the time I finished speaking and the time the click is processed. In Iris when I want to click freely on the screen with a delayed right-click macro, I need to time in my eye movements just right with the macro to work.

Managing that latency has always been a challenge for me. I'm wondering if a facial expression in Microsoft Kinect for Windows can be recognized faster than a voice command in voice attack.

What is the latency between an expression and the input? Do you have that measured? If not, is there a video that shows someone's face and the effects on the screen?
JoyfulRogue
Posts: 10
Joined: Fri May 22, 2015 12:07 pm

Re: What the latency between facial expression and mouseclick?

Post by JoyfulRogue »

Oh, and from what I read that you actually don't support kinect for Windows?

Is there an easy way to connect the Xbox one version to my computer?If so, from what I hear it shouldn't work very well when I sit this close to the monitor without the modifications implemented in the Windows version that make it work better short distance
User avatar
Xcessity
Site Admin
Posts: 284
Joined: Sat May 16, 2015 2:23 pm
Location: Graz
Contact:

Re: What the latency between facial expression and mouseclick?

Post by Xcessity »

Hi,

concerning the latency of the KM:

The KinesicMouse can track your face at up to 30 fps (60 fps in IR mode using Intel RealSense camera). So let's say you have a decent CPU and you are using the full 30 fps. So the worst delay you can have is 33 ms and usually about half, so 16 ms delay.

This is pretty fast and there are no facial expressions where you will notice any delay. The only thing where you will benefit from higher frames rates is the "relative mouse mode". In this mode your head rotations are directly translated into a mouse cursor position. Since a modern screen is 1920 pixels wide and a comfortable rotation range is only 40 degrees this tracking needs to be very accurate. This specific mouse mode actually needs to calculate a mean value from a few frames (4-5) to get a stable mouse cursor on screen.

Besides that everything else is very quick and way better than voice commands.


Cencerning the Kinect ONE sensor:
To connect the Xbox ONE Kinect to a PC you actually need the "Kinect for Windows Adapter" from Microsoft. It costs about 50 EUR and is available at amzon or the official Microsoft Store. There must be at least 50cm between the Kinect and your face otherwise it wont be able measure depth.

If you are looking for a more close range sensor I would recommend the external Intel RealsSense camera F200. It works at a lower range, but gets way worse on distances greater than 60cm. So in terms of range and flexibility I would favor the Kinect ONE.
Post Reply