Investigate using other AIs #78

Activity

1. Kariaro commented on Apr 10, 2023

   

   Kariaro

   on Apr 10, 2023

   OwnerAuthor

   More actions

   After testing https://github.com/creativeIKEP/HolisticBarracuda
   I found some pros and cons:

   • Much better CPU usage
     • By doing computations on the GPU the cpu usage goes from 30% to 10%
       But this also reduces the framerate to 20 fps instead of 100 fps
   • Worse face tracking
     • The face tracking is good but at 1 meter from the screen the
       high detail face tracking stops working and freezes
   • Very easy to switch to
     • It is easy to install and the API is very easy to follow
       Converting it to the current project would not be hard

   Before I make any changes I would like to ask the maintainer if it is possible
   to fix the face tracking and increase the framerate.
2. Kariaro commented on May 9, 2023

   

   Kariaro

   on May 9, 2023

   OwnerAuthor

   More actions

   I've found a way to implement these models myself but there are some problems.

   • The standard MediaPipe calculator uses a lot of code that includes some pre/post processing steps.
     Without these steps the output becomes significantly more jittery and random and will not be good enough for mocap.
     Therefore, for each AI model I would need to reimplement all the code from each calculator and make sure everything is still working properly.
   • I'm not sure how much performance would be saved from using these models because of the amount of pre/post processing steps. I'm going to have to learn more about this before I can say how long it would take to implement.

   Though with enough time this method would increase the performance of the app by a lot but it would take a really long time to implement and test.

   👍React with 👍1Reacted by HardCoded
4. Kariaro commented on Jun 12, 2023

   

   Kariaro

   on Jun 12, 2023

   Last edited by Kariaro

   OwnerAuthor

   More actions

   I think there's no reason to implement the HolisticBarracuda replacing the good-old MediaPipe, just look how good the code and MediaPipe.

   Maybe, you need to optimize the code only, since MediaPipe doesn't always the reason for low performance.

   Most of the performance loss comes from running MediaPipe on the CPU.

   The problem with MediaPipe is that I cannot run it on the GPU. For low end computers this makes the app hard to use. (If you are playing a game that also uses a lot of CPU)

   HolisticBarracuda runs on GPU and is multiplatform but the problem I have with HolisticBarracuda is that it’s not async meaning that it slows down the render thread. (Low CPU but terrible fps)

   This is why I want to replace MediaPipe.

   If MediaPipe had support for GPU I would keep using it though.