KinesicMouse with Android/no iPhone

[benedikt]

Hello,

I first had tried Iris and it works fantastically. Then I wanted to also try KinesicMouse however I have no iPhone. There were other posts asking for an android version.
So after searching the internet i found a few alternatives:

• Face Mocap.
  https://play.google.com/store/apps/details?id=mx.motion.ar.facemocap
  Though my phone wasn't compatible. You will need arcore. Here is a list of compatible phones.
  https://developers.google.com/ar/devices?hl=de#google_play_devices

• MeFaMo
  https://github.com/JimWest/MeFaMo
  This one uses any webcam even an internal. Although you have to build it yourself with Python.
  I tried it and its working. Maybe not as great as with FaceId but not bad.
  I will give installation instruction for those not familar with programming.

• Download python https://www.python.org/ftp/python/3.10.10/python-3.10.10-amd64.exe. Open it.
  Make sure you check add python 3.10 to Path. Click customize install. Check pip. Then next. Check Add Python to enviroment variables. Then install

• Go to https://github.com/JimWest/MeFaMo .

• Click on the green button labeled code then download zip.

• Extract the folder then go to ...\Downloads\MeFaMo-main\. hold Shift then right click the only folder.
  An option open with powershell appears. Click it. Write or copy&paste python setup.py install. Enter. Wait until it finishes.

• write python mefamo_cli.py --ip "your computers ip". Enter. The webcam should activate. Then the smiley in KinesicMouse turns green

I hope it helps someone.

[Xcessity]

Hello Benedikt,

thank you for the suggestions. I have tried all Android alternatives and various face tracking packages based on webcam. There is nothing that comes even close to the accuracy and amount of signals as the Apple FaceID. It has the benefit of adding 3D depth data to the face tracking features which others don't have. I don't say that because I am a big fan of Apple, they just bought the company (former faceshift) that had the best face tracking on the market.

I know there are many people using Android and my suggestion is to just get a used iPhone X. You can even get one with a broken screen/back for around 50 USD. Just treat the phone as a dedicated 3D camera. They are cheaper now than the former 3D cameras (Kinect, Realsense or Orbbec) and also offload the processing from your PC to the phone. In this scenario you don't need a mobile plan or sim card either and you have your usual mobile phone ready for other stuff.

Just my opinion on this topic, maybe I am biased, as I am the developer of the software ;)

Best,
M

[TomIsHere]

To update regarding both of your messages:

Thank you to the user for suggesting MeFaMo and fortunately the program does not require building the project from its source code. It has an executable file you may run instead found on the Releases page.

Steps to set up:
0. Connect a camera to your computer.
1. Go to https://github.com/JimWest/MeFaMo/tags
2. Download ZIP folder
3. Unzip the folder
4. Run the executable file.
5. Wait a while. It'll eventually show a preview of your face and the system working.
6. Start Kinesic Mouse LIVE and click Resume

Results:
Unfortunately, in my testing with an HD 1080p Logitech camera the MeFaMo software is very unreliable and would randomly click even when I was not smiling or making any movements. I reviewed my settings but the issue persisted.

As for using my iPhone 12 Pro with Kinesic Mouse LIVE there are 3 problems with this
1. My iPhone overheated when running the app for ~30 minutes and using default settings
2. Over time it would drain/harm the iPhone battery. So really only worthwhile on a used iPhone you use solely for this purpose.
3. I'm looking for a solution I can use at work and home. When I'm at work, however, my employer does not allow us to connect phones to the same network as our computer is on so I cannot use this.

If the developer of Kinesic Mouse LIVE has any suggestions or advice I'd greatly appreciate it!

[Xcessity]

Hello TomIsHere,

thank you for the input on MeFaMo. Unfortunately it confirms my initial thoughts on 2D cam based face trackers.

On to your remarks:

1. My iPhone overheated when running the app for ~30 minutes and using default settings

The "overheating" sounds worse than it really is. The iPhone just throttles the fps from 60 to 30fps. It remains in a stable state running on 30fps which is sufficient for good control. If you want to keep the 60fps there are heat sinks available for phones. Here is a discussion about the heat sinks some users have tried with the iPhone running the KinesicMouse.

2. Over time it would drain/harm the iPhone battery. So really only worthwhile on a used iPhone you use solely for this purpose.

If you are worried about battery life you can always keep the power cable attached. Modern phones & laptops will draw the power from the cable instead of the battery. Still I think a dedicated phone makes a lot of sense. You can leave it at your setup ready to roll when you need it.

3. I'm looking for a solution I can use at work and home. When I'm at work, however, my employer does not allow us to connect phones to the same network as our computer is on so I cannot use this.

You can connect the iPhone directly to your PC using USB. You can still access the internet through the WLAN connection provided by your work place. You can watch a tutorial video on how to connect with USB on youtube: https://youtu.be/_rdE-NxkUVw

Hope this helps,
Best,
M

[TomIsHere]

Thank you for your reply and tips!

I've given it a second chance and tested the software again. Unfortunately, I've decided to not use the software for now. My hands hurt too much rn to go into detail but in short because I was trying to replace my mouse with this software but head tracking software is apparently not precise enough to accurately click small buttons on the screen without having to try many times. Also in FPS games (Rainbow Six Siege to be precise) head tracking to replace a mouse is not as good as I expected it to be- it moves too much and gets disorienting/hard to aim. Even when I'm still the cursor still is moving a bit so I cannot accurately aim.

Thank you for making this software though and if my hand problems get worse maybe I'll come back to it eventually.