Getting Your Hands Dirty with TensorFlow 2.0 and Keras API

Getting Your Hands Dirty with TensorFlow 2.0 and Keras API

Diving into technical details of the regression model creation with TensorFlow 2.0 and Keras API. In TensorFlow 2.0, Keras comes out of the box with TensorFlow library. API is simplified and more convenient to use.

Diving into technical details of the regression model creation with TensorFlow 2.0 and Keras API. In TensorFlow 2.0, Keras comes out of the box with TensorFlow library. API is simplified and more convenient to use.

TensorFlow 2.0 comes with Keras packaged inside, there is no need to import Keras as a separate module (although you can do this if you need). TensorFlow 2.0 API is simplified and improved. This is good news for us — Machine Learning developers.

This is how you import Keras now, from TensorFlow:

from tensorflow import feature_column
from tensorflow import keras
from tensorflow.keras import layersprint(tf.__version__)
2.0.0

I’m using tf.data input pipeline to encode categorical columns, Keras API works well with tf.data. One of the main advantages of tf.data is the fact that it acts as a bridge between data and model. There is no need to transform data by yourself, enough to define transformation rule — transformed data automatically will be applied during training.

Data is fetched from CSV file into Pandas dataframe:

column_names = ['report_id','report_params','day_part','exec_time']
raw_dataframe = pd.read_csv('report_exec_times.csv')
dataframe = raw_dataframe.copy()dataframe.head()

Column values for report_params vary, we need to normalize this column (make values to be on a similar scale):

eps=0.001 # 0 => 0.1¢
dataframe['report_params'] = np.log(dataframe.pop('report_params')+eps)

I’m using a utility method (this method is taken from TensorFlow tutorial) to create tf.data dataset from Pandas dataframe:

def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('exec_time')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

Next, we need to define data mapping for categorical columns encoding. I’m using TensorFlow vocabulary list function, including mapping for all unique values (if there are many values, it would be better to use embedding API). Two columns are encoded — report_id and day_part:

feature_columns = []feature_columns.append(feature_column.numeric_column('report_params'))report_id = feature_column.categorical_column_with_vocabulary_list('report_id', ['1', '2', '3', '4', '5'])
report_id_one_hot = feature_column.indicator_column(report_id)
feature_columns.append(report_id_one_hot)day_part = feature_column.categorical_column_with_vocabulary_list('day_part', ['1', '2', '3'])
day_part_one_hot = feature_column.indicator_column(day_part)
feature_columns.append(day_part_one_hot)

Create Keras dense features layer out of array with TensorFlow encodings. We will use this layer during Keras model construction to define model training features:

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)

We are done with features. Next, convert Pandas dataframe to tf.data with the help of utility function:

batch_size = 32
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

Dense features layer is used when Keras sequential model is defined (there is no need to pass array of features later into a fit function):

def build_model(feature_layer):
  model = keras.Sequential([
    feature_layer,
    layers.Dense(16, activation='relu'),
    layers.Dense(16, activation='relu'),
    layers.Dense(1)
  ])  optimizer = keras.optimizers.RMSprop(0.001)  model.compile(loss='mse',
                optimizer=optimizer,
                metrics=['mae', 'mse'])
  return model

Training is executed through model.fit function. We are using tf.data input pipeline to pass training and validation sets:

history = model.fit(train_ds,
              validation_data=val_ds,
              epochs=EPOCHS,
              callbacks=[early_stop])

The awesome thing about it — data encoding happens behind the scenes, based on the rules defined for the feature layer.

How to run model.predict function with tf.data and feature layer? It’s simple.

Construct Pandas dataframe with input data:

headers = ['report_id', 'report_params', 'day_part']
dataframe_input = pd.DataFrame([[1, 15, 3]],
                                columns=headers, 
                                dtype=float,
                                index=['input'])

Convert report_params value to be on the same scale as it was for the training:

eps=0.001 # 0 => 0.1¢
dataframe_input['report_params'] = np.log(dataframe_input.pop('report_params')+eps)

Create tf.data input pipeline out of Pandas dataframe:

input_ds = tf.data.Dataset.from_tensor_slices(dict(dataframe_input))
input_ds = input_ds.batch(1)

Run model.predict function:

res = model.predict(input_ds)
print(res)

Resources:

  • Source code with sample data is available on my GitHub repo

Enjoy !

Angular 9 Tutorial: Learn to Build a CRUD Angular App Quickly

What's new in Bootstrap 5 and when Bootstrap 5 release date?

Brave, Chrome, Firefox, Opera or Edge: Which is Better and Faster?

How to Build Progressive Web Apps (PWA) using Angular 9

What is new features in Javascript ES2020 ECMAScript 2020

Machine Learning, Data Science and Deep Learning with Python

Complete hands-on Machine Learning tutorial with Data Science, Tensorflow, Artificial Intelligence, and Neural Networks. Introducing Tensorflow, Using Tensorflow, Introducing Keras, Using Keras, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Learning Deep Learning, Machine Learning with Neural Networks, Deep Learning Tutorial with Python

Machine Learning Tutorial with Python, Jupyter, KSQL and TensorFlow

Machine Learning With Python, Jupyter, KSQL, and TensorFlow. This post focuses on how the Kafka ecosystem can help solve the impedance mismatch between data scientists, data engineers and production engineers.

TensorFlow Vs PyTorch: Comparison of the Machine Learning Libraries

Libraries play an important role when developers decide to work in Machine Learning or Deep Learning researches. In this article, we list down 10 comparisons between TensorFlow and PyTorch these two Machine Learning Libraries.