The TensorFlow Keras Summary Capture Layer

The TensorFlow Keras Summary Capture Layer

How to Capture and Record Arbitrary Tensors in TensorFlow 2

In previous posts, I have told you about how my team at Mobileye, (officially known as Mobileye, an Intel Company), has tackled some of the challenges that came up, while using TensorFlow to train deep neural networks. In particular, I have covered topics such as performance profiling, and debugging. This post addresses an additional component of training machine learning models, that of monitoring the learning process.

Monitoring the learning process is an important, and often time-consuming, part of DNN training, during which we track a variety of tensors, metrics and statistics, in order to understand how our training is progressing, and figure out what improvements to make to model architecture and/or hyperparameters.

One of the most popular ways of doing this when training in TensorFlow, is using TensorBoard. TensorBoard supports a variety of methods for tracking and debugging the training process. On our team, we rely heavily on TensorBoard. We track losses, generate gradient histograms, and measure activation outputs. We log metrics, display confusion matrices, and generate visual images from the output data. Sometimes we use it debug intermediate operations performed by the loss function, or to measure the distribution of weights on a specific layer in the graph. We use TensorBoard extensively.

The way to use TensorBoard is by inserting “summary” commands into the training code using the tf.summary module. In version 2, TensorFlow made significant revisions to the training flow, and, in particular, to the tf summary mechanism. While this revision simplified some of the straightforward usages of tf.summary, some of the more advanced usages are not immediately obvious. For example, it is not at all clear (to me, at least), how TensorFlow intended for me to be able to log graph (non-eager) tensors, such as the distribution of the output of a specific activation layer within my graph, or the value of an arbitrary tensor within my loss function. (If you know, please don’t hesitate to share :).)

tensorflow machine-learning keras

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Hire Machine Learning Engineer | Offshore Machine Learning Experts

We are a Machine Learning Services provider offering custom AI solutions, Machine Learning as a service & deep learning solutions. Hire Machine Learning experts & build AI Chatbots, Neural networks, etc. 16+ yrs & 2500+ clients.

5 Latest Technology Trends of Machine Learning for 2021

Check out the 5 latest technologies of machine learning trends to boost business growth in 2021 by considering the best version of digital development tools. It is the right time to accelerate user experience by bringing advancement in their lifestyle.

Hire Machine Learning Developers in India

We supply you with world class machine learning experts / ML Developers with years of domain experience who can add more value to your business.

Applications of machine learning in different industry domains

We supply you with world class machine learning experts / ML Developers with years of domain experience who can add more value to your business.

Hire Machine Learning Developer | Hire ML Experts in India

We supply you with world class machine learning experts / ML Developers with years of domain experience who can add more value to your business.