Initially available under the ML Kit early access program, Pose Detection is now officially part of ML Kit. The library is capable of tracking the human body, including facial landmarks, hands, and feet.

ML Kit Pose Detection API is based on Google’s BlazePose pipeline, which combines computer vision and machine learning to infer 33 two-dimensional body landmarks. This represents a significant improvement over the current standard for body pose, which uses the COCO dataset for keypoint detection, according to Google.

The COCO keypoints only localize to the ankle and wrist points, lacking scale and orientation information for hands and feet, which is vital for practical applications like fitness and dance. The inclusion of more keypoints is crucial for the subsequent application of domain-specific pose estimation models, like those for hands, face, or feet.

BlazePose achieves this result by building on top of the previously available BlazeFace and BlazePalm topologies used to create face and hand models. BlazePose achieves real-time performance on mobile phones when using only CPU inference, while using GPU inference makes it also possible to run subsequent ML models for face or hand tracking.

#google #android #mobile #computer vision #ios #machine learning #development #news