Preparing a TensorFlow Model to Run Inference on an i.mx8M Plus NPU
In this article, we will explain in this article which steps you have to take to transform and quantize your model with different TensorFlow versions. We are only looking into post training quantization.
phyBOARD pollux [Image via phytec.de under license to Jan Werth]
i.mx8MPlus block diagram from NXP [Image via phytec.de, with permission from NXP]
As the neural processing unit (NPU) from NXP need a fully int8 quantized model we have to look into full int8 quantization of a TensorFlow lite or PyTorch model. Both libraries are supported with the eIQ library from NXP. Here we will only look into the TensorFlow variant.
The general overview on how to do post training quantization can be found on the TensorFlow website.
Tensorflow Lite is one of my favourite software packages. It enables easy and fast deployment on a range of hardware and now comes with a wide range of delegates to accelerate inference — GPU, Core ML and Hexagon, to name a few. One drawback of Tensorflow Lite however is that it’s been designed with mobile applications in mind, and therefore isn’t optimised for Intel & AMD x86 processors. Better x86 support is on the Tensorflow Lite development roadmap, but for now Tensorflow Lite mostly relies on converting ARM Neon instructions to SSE via the Neon_2_SSE bridge
With the launch of TensorFlow Lite, embedding machine learning models in Android apps has never been easier. TensorFlow lite provides a
We’re going to discuss how to implement a housing price prediction machine learning model for mobile using TensorFlow Lite. We’ll learn how to train a TensorFlow Lite neural network for regression that provides a continuous value prediction, specifically in the context of housing prices.
This article will explain how to reduce the size of an image classification machine learning model for mobile using TensorFlow Lite, in order to make it fit and work on mobile devices. Predicting whether a person in an image is wearing a mask or not. Training an Image Classification Model for Mobile using TensorFlow Lite
A Definitive Guide for Audio Processing in Android with TensorFlow Lite Models. Explanation with a demo app implementatio. This guide describes how to process audio files in Android, in order to feed them into deep learning models built using TensorFlow.