In 2018 Google announced the TensorFlow release of PoseNet, a machine learning model capable of detecting persons in images and estimating the location of their body parts, a technique known as pose estimation.

Poses estimated by the PoseNet algorithm running on the Coral Edge TPU accelerator. On the right, an overlay of the keypoint heatmaps that are not normally accessible in the Edge TPU model. Original photo by Jaisingh rathore /CC BY 2.5.

The PoseNet detection framework is available for JavaScript (TensorFlow.js), Android/iOS mobile devices (TensorFlow Lite), and Edge TPU accelerator (Google Coral).

Lately, I have been mostly interested in the Edge TPU version of the framework since I have been working on several projects that involve people detection and tracking “in the wild”, and the USB version of the TPU makes it possible to perform real-time pose estimation on small embedded devices such as the Raspberry Pi or the STM32P1 platform.

Limitations of PoseNet for Edge TPU

The Coral engineers have done a great job at packaging the code and models in a way that makes it easy for the developer to get up and running with the PoseNet framework. To simplify usage, however, some of the model parameters have been hardcoded. For example, the number people/poses that can be detected in an image has been limited to 10. Other parameters of the decoding algorithm have also been hardcoded, and while I found the default values to work well in many situations, I think there is an opportunity to improve the pose estimation by fine-tuning some of the hidden parameters. Please read on if you are interested in the details.

#edge-tpu #machine-learning-ai #pose-estimation #computer-v