Yvette  Bell

Yvette Bell

1641758400

Keras + Hyperopt: Convenient Hyperparameter Optimization

Hyperas 

Hyperas brings fast experimentation with Keras and hyperparameter optimization with Hyperopt together. It lets you use the power of hyperopt without having to learn the syntax of it. Instead, just define your keras model as you are used to, but use a simple template notation to define hyper-parameter ranges to tune.

Installation

pip install hyperas

Quick start

Assume you have data generated as such

def data():
    x_train = np.zeros(100)
    x_test = np.zeros(100)
    y_train = np.zeros(100)
    y_test = np.zeros(100)
    return x_train, y_train, x_test, y_test

and an existing keras model like the following

def create_model(x_train, y_train, x_test, y_test):
    model = Sequential()
    model.add(Dense(512, input_shape=(784,)))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(10))
    model.add(Activation('softmax'))

    # ... model fitting

    return model

To do hyper-parameter optimization on this model, just wrap the parameters you want to optimize into double curly brackets and choose a distribution over which to run the algorithm.

In the above example, let's say we want to optimize for the best dropout probability in both dropout layers. Choosing a uniform distribution over the interval [0,1], this translates into the following definition. Note that before returning the model, to optimize, we also have to define which evaluation metric of the model is important to us. For example, in the following, we optimize for accuracy.

Note: In the following code we use 'loss': -accuracy, i.e. the negative of accuracy. That's because under the hood hyperopt will always minimize whatever metric you provide. If instead you want to actually want to minimize a metric, say MSE or another loss function, you keep a positive sign (e.g. 'loss': mse).

from hyperas.distributions import uniform

def create_model(x_train, y_train, x_test, y_test):
    model = Sequential()
    model.add(Dense(512, input_shape=(784,)))
    model.add(Activation('relu'))
    model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense(10))
    model.add(Activation('softmax'))

    # ... model fitting

    score = model.evaluate(x_test, y_test, verbose=0)
    accuracy = score[1]
    return {'loss': -accuracy, 'status': STATUS_OK, 'model': model}

The last step is to actually run the optimization, which is done as follows:

best_run = optim.minimize(model=create_model,
                          data=data,
                          algo=tpe.suggest,
                          max_evals=10,
                          trials=Trials())

In this example we use at most 10 evaluation runs and the TPE algorithm from hyperopt for optimization.

Check the "complete example" below for more details.

Complete example

Note: It is important to wrap your data and model into functions as shown below, and then pass them as parameters to the minimizer. data() returns the data the create_model() needs. An extended version of the above example in one script reads as follows. This example shows many potential use cases of hyperas, including:

  • Varying dropout probabilities, sampling from a uniform distribution
  • Different layer output sizes
  • Different optimization algorithms to use
  • Varying choices of activation functions
  • Conditionally adding layers depending on a choice
  • Swapping whole sets of layers
from __future__ import print_function
import numpy as np

from hyperopt import Trials, STATUS_OK, tpe
from keras.datasets import mnist
from keras.layers.core import Dense, Dropout, Activation
from keras.models import Sequential
from keras.utils import np_utils

from hyperas import optim
from hyperas.distributions import choice, uniform


def data():
    """
    Data providing function:

    This function is separated from create_model() so that hyperopt
    won't reload data for each evaluation run.
    """
    (x_train, y_train), (x_test, y_test) = mnist.load_data()
    x_train = x_train.reshape(60000, 784)
    x_test = x_test.reshape(10000, 784)
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    x_train /= 255
    x_test /= 255
    nb_classes = 10
    y_train = np_utils.to_categorical(y_train, nb_classes)
    y_test = np_utils.to_categorical(y_test, nb_classes)
    return x_train, y_train, x_test, y_test


def create_model(x_train, y_train, x_test, y_test):
    """
    Model providing function:

    Create Keras model with double curly brackets dropped-in as needed.
    Return value has to be a valid python dictionary with two customary keys:
        - loss: Specify a numeric evaluation metric to be minimized
        - status: Just use STATUS_OK and see hyperopt documentation if not feasible
    The last one is optional, though recommended, namely:
        - model: specify the model just created so that we can later use it again.
    """
    model = Sequential()
    model.add(Dense(512, input_shape=(784,)))
    model.add(Activation('relu'))
    model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense({{choice([256, 512, 1024])}}))
    model.add(Activation({{choice(['relu', 'sigmoid'])}}))
    model.add(Dropout({{uniform(0, 1)}}))

    # If we choose 'four', add an additional fourth layer
    if {{choice(['three', 'four'])}} == 'four':
        model.add(Dense(100))

        # We can also choose between complete sets of layers

        model.add({{choice([Dropout(0.5), Activation('linear')])}})
        model.add(Activation('relu'))

    model.add(Dense(10))
    model.add(Activation('softmax'))

    model.compile(loss='categorical_crossentropy', metrics=['accuracy'],
                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})

    result = model.fit(x_train, y_train,
              batch_size={{choice([64, 128])}},
              epochs=2,
              verbose=2,
              validation_split=0.1)
    #get the highest validation accuracy of the training epochs
    validation_acc = np.amax(result.history['val_acc']) 
    print('Best validation acc of epoch:', validation_acc)
    return {'loss': -validation_acc, 'status': STATUS_OK, 'model': model}


if __name__ == '__main__':
    best_run, best_model = optim.minimize(model=create_model,
                                          data=data,
                                          algo=tpe.suggest,
                                          max_evals=5,
                                          trials=Trials())
    X_train, Y_train, X_test, Y_test = data()
    print("Evalutation of best performing model:")
    print(best_model.evaluate(X_test, Y_test))
    print("Best performing model chosen hyper-parameters:")
    print(best_run)

FAQ

Here is a list of a few popular errors

TypeError: require string label

You're probably trying to execute the model creation code, with the templates, directly in python. That fails simply because python cannot run the templating in the braces, e.g. {{uniform..}}. The def create_model(...) function is in fact not a valid python function anymore.

You need to wrap your code in a def create_model(...): ... function, and then call it from optim.minimize(model=create_model,... like in the example.

The reason for this is that hyperas works by doing template replacement of everything in the {{...}} into a separate temporary file, and then running the model with the replaced braces (think jinja templating).

This is the basis of how hyperas simplifies usage of hyperopt by being a "very simple wrapper".

TypeError: 'generator' object is not subscriptable

This is currently a known issue.

Just pip install networkx==1.11

NameError: global name 'X_train' is not defined

Maybe you forgot to return the x_train argument in the def create_model(x_train...) call from the def data(): ... function.

You are not restricted to the same list of arguments as in the example. Any arguments you return from data() will be passed to create_model()

notebook adjustment

If you find error like "No such file or directory" or OSError, Err22, you may need add notebook_name='simple_notebook'(assume your current notebook name is simple_notebook) in optim.minimize function like this:

best_run, best_model = optim.minimize(model=model,
                                      data=data,
                                      algo=tpe.suggest,
                                      max_evals=5,
                                      trials=Trials(),
                                      notebook_name='simple_notebook')

How does hyperas work?

All we do is parse the data and model templates and translate them into proper hyperopt by reconstructing the space object that's then passed to fmin. Most of the relevant code is found in optim.py and utils.py.

How to read the output of a hyperas model?

Hyperas translates your script into hyperopt compliant code, see here for some guidance on how to interpret the result.

How to pass arguments to data?

Suppose you want your data function take an argument, specify it like this using positional arguments only (not keyword arguments):

import pickle
def data(fname):
    with open(fname,'rb') as fh:
        return pickle.load(fh)

Note that your arguments must be implemented such that repr can show them in their entirety (such as strings and numbers). If you want more complex objects, use the passed arguments to build them inside the data function.

And when you run your trials, pass a tuple of arguments to be substituted in as data_args:

best_run, best_model = optim.minimize(
    model=model,
    data=data,
    algo=tpe.suggest,
    max_evals=64,
    trials=Trials(),
    data_args=('my_file.pkl',)
)

What if I need more flexibility loading data and adapting my model?

Hyperas is a convenience wrapper around Hyperopt that has some limitations. If it's not convenient to use in your situation, simply don't use it -- and choose Hyperopt instead. All you can do with Hyperas you can also do with Hyperopt, it's just a different way of defining your model. If you want to squeeze some flexibility out of Hyperas anyway, take a look here.

Running hyperas in parallel?

You can use hyperas to run multiple models in parallel with the use of mongodb (which you'll need to install and setup users for). Here's a short example using MNIST:

Copy and modify examples/mnist_distributed.py (bump up max_evals if you like):

Run python mnist_distributed.py. It will create a temp_model.py file. Copy this file to any machines that will be evaluating models. It will then begin waiting for evaluation results

On your other machines (make sure they have a python installed with all your dependencies, ideally with the same versions) run:

export PYTHONPATH=/path/to/temp_model.py
hyperopt-mongo-worker --exp-key='mnist_test' --mongo='mongo://username:pass@mongodb.host:27017/jobs'

Once max_evals have been completed, you should get an output with your best model. You can also look through your mongodb and examine the results, to get the best model out and run it, do:

from pymongo import MongoClient
from keras.models import load_model
import tempfile
c = MongoClient('mongodb://username:pass@mongodb.host:27017/jobs')
best_model = c['jobs']['jobs'].find_one({'exp_key': 'mnist_test'}, sort=[('result.loss', -1)])
temp_name = tempfile.gettempdir()+'/'+next(tempfile._get_candidate_names()) + '.h5'
with open(temp_name, 'wb') as outfile:
    outfile.write(best_model['result']['model_serial'])
model = load_model(temp_name)

Author: maxpumperla
Source Code: https://github.com/maxpumperla/hyperas
License: MIT License

#python #keras 

What is GEEK

Buddha Community

Keras + Hyperopt: Convenient Hyperparameter Optimization

Keras Tutorial - Ultimate Guide to Deep Learning - DataFlair

Welcome to DataFlair Keras Tutorial. This tutorial will introduce you to everything you need to know to get started with Keras. You will discover the characteristics, features, and various other properties of Keras. This article also explains the different neural network layers and the pre-trained models available in Keras. You will get the idea of how Keras makes it easier to try and experiment with new architectures in neural networks. And how Keras empowers new ideas and its implementation in a faster, efficient way.

Keras Tutorial

Introduction to Keras

Keras is an open-source deep learning framework developed in python. Developers favor Keras because it is user-friendly, modular, and extensible. Keras allows developers for fast experimentation with neural networks.

Keras is a high-level API and uses Tensorflow, Theano, or CNTK as its backend. It provides a very clean and easy way to create deep learning models.

Characteristics of Keras

Keras has the following characteristics:

  • It is simple to use and consistent. Since we describe models in python, it is easy to code, compact, and easy to debug.
  • Keras is based on minimal substructure, it tries to minimize the user actions for common use cases.
  • Keras allows us to use multiple backends, provides GPU support on CUDA, and allows us to train models on multiple GPUs.
  • It offers a consistent API that provides necessary feedback when an error occurs.
  • Using Keras, you can customize the functionalities of your code up to a great extent. Even small customization makes a big change because these functionalities are deeply integrated with the low-level backend.

Benefits of using Keras

The following major benefits of using Keras over other deep learning frameworks are:

  • The simple API structure of Keras is designed for both new developers and experts.
  • The Keras interface is very user friendly and is pretty optimized for general use cases.
  • In Keras, you can write custom blocks to extend it.
  • Keras is the second most popular deep learning framework after TensorFlow.
  • Tensorflow also provides Keras implementation using its tf.keras module. You can access all the functionalities of Keras in TensorFlow using tf.keras.

Keras Installation

Before installing TensorFlow, you should have one of its backends. We prefer you to install Tensorflow. Install Tensorflow and Keras using pip python package installer.

Starting with Keras

The basic data structure of Keras is model, it defines how to organize layers. A simple type of model is the Sequential model, a sequential way of adding layers. For more flexible architecture, Keras provides a Functional API. Functional API allows you to take multiple inputs and produce outputs.

Keras Sequential model

Keras Functional API

It allows you to define more complex models.

#keras tutorials #introduction to keras #keras models #keras tutorial #layers in keras #why learn keras

Aayush Singh

Aayush Singh

1607579145

Keras Tutorial For Beginners | What is Keras | Keras Sequential Model | Keras Training

In this video on Keras, you will understand what is Keras and why do we need it, how to compose different models in Keras like the Sequential model and functional model, and later on how to define the inputs, how to connect layers over, and finally hands-on demo.
Why Keras is important

Keras is an Open Source Neural Network library written in Python that runs on top of Theano or Tensorflow. It is designed to be modular, fast, and easy to use. Keras is very quick to make a network model. If you want to make a simple network model with a few lines, Keras can help you with that.

Call Our Course Advisors IND: +91-7022374614 US: 1-800-216-8930 (Toll-Free) sales@intellipaat.com
Link: https://www.youtube.com/watch?v=nS1J-2uoKto

#keras tutorial for beginners #what is keras #keras sequential model #keras training

Why Learn Keras - Reasons Why Choose Keras - DataFlair

This article is the spotlight on the need for python deep learning library, Keras. Keras offers a uniform face for various deep learning frameworks including Tensorflow, Theano, and MXNet. Let us see why you should choose and learn keras now.

Why learn Keras

Why Learn Keras?

Keras makes deep learning accessible and local on your computer.It also acts as a frontend for other big cloud providers. It is the most voted recommendation for beginners who want to start their journey in machine learning. It provides a minimal approach to run neural networks. This allows students to learn complex features from input data sequentially.

Features of Keras

Let us see some of the features of keras that make you learn Keras.

1. Simple API

Keras is the most easy to use the library for machine learning for beginners. Being simple helps it to bring machine learning from imaginations to reality. It provides an infrastructure that can be learned in very less time. Using Keras, you will be able to stack layers like experts.

2. Pythonic Nature

Python is the most popular library for machine learning and Data Science. The compatibility with python allows Keras to have many useful features. Writing less code, easy to debug, easy to deploy, extensibility is due to the support of Keras with python 2.7 and python 3.6.

3. Strong Backend Support

Keras being a high-level API provides support for multiple popular and powerful backend frameworks. Tensorflow, theano, CNTK are very dominant for backend computations and Keras supports all of them.

4. Base for Innovations

The importance of Keras leads to many other innovative tools to explore deep learning. These tools are built on top of Keras making Keras as the base. The following tools are:

  • Deepjazz: This is deep learning-driven jazz built using Keras and theano, available on github.
  • Eclipse Picasso: It is a visualization tool that works with Keras checkpoints.
  • Auto Keras: It is built upon Keras and used for machine learning model automation.

Reasons to Learn and Use Keras

  • Keras allows us to switch between the backends as per the requirement of our applications. It acts as a wrapper that gives us the privilege to use either TensorFlow, theano, or any other framework.
  • Keras is very easy and enjoyable to use. It uses great guiding principles like extensibility, python nativeness, and modularity.
  • The ability of Keras to create the state of the art implementations of common deep neural networks. These are fast and it is easy to get them running using Keras.
  • Being Keras user, you will be more faster and productive, you will have the ability to try more ideas.
  • Keras provides Multi-GPU and strong distributed support. We can run our deep learning models on large GPU clusters.
  • We can deploy Keras deep learning models on multiple platforms. For example, We can deploy in the browser using tensorflow.js, on the server using either TensorFlow serving or using Node.js runtime. On mobile devices i.e in android or IOS, we can deploy using TensorFlow Lite.
  • Keras has a large ecosystem of products to support your deep learning development. Some of the popular products are Tensorflow Cloud, Keras Tuner, Tensorflow Lite,Tensorflow.js, and Tensorflow Model Optimizatio

#keras tutorials #importance of keras #keras features #learn keras #deep learning

Yvette  Bell

Yvette Bell

1641758400

Keras + Hyperopt: Convenient Hyperparameter Optimization

Hyperas 

Hyperas brings fast experimentation with Keras and hyperparameter optimization with Hyperopt together. It lets you use the power of hyperopt without having to learn the syntax of it. Instead, just define your keras model as you are used to, but use a simple template notation to define hyper-parameter ranges to tune.

Installation

pip install hyperas

Quick start

Assume you have data generated as such

def data():
    x_train = np.zeros(100)
    x_test = np.zeros(100)
    y_train = np.zeros(100)
    y_test = np.zeros(100)
    return x_train, y_train, x_test, y_test

and an existing keras model like the following

def create_model(x_train, y_train, x_test, y_test):
    model = Sequential()
    model.add(Dense(512, input_shape=(784,)))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(10))
    model.add(Activation('softmax'))

    # ... model fitting

    return model

To do hyper-parameter optimization on this model, just wrap the parameters you want to optimize into double curly brackets and choose a distribution over which to run the algorithm.

In the above example, let's say we want to optimize for the best dropout probability in both dropout layers. Choosing a uniform distribution over the interval [0,1], this translates into the following definition. Note that before returning the model, to optimize, we also have to define which evaluation metric of the model is important to us. For example, in the following, we optimize for accuracy.

Note: In the following code we use 'loss': -accuracy, i.e. the negative of accuracy. That's because under the hood hyperopt will always minimize whatever metric you provide. If instead you want to actually want to minimize a metric, say MSE or another loss function, you keep a positive sign (e.g. 'loss': mse).

from hyperas.distributions import uniform

def create_model(x_train, y_train, x_test, y_test):
    model = Sequential()
    model.add(Dense(512, input_shape=(784,)))
    model.add(Activation('relu'))
    model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense(10))
    model.add(Activation('softmax'))

    # ... model fitting

    score = model.evaluate(x_test, y_test, verbose=0)
    accuracy = score[1]
    return {'loss': -accuracy, 'status': STATUS_OK, 'model': model}

The last step is to actually run the optimization, which is done as follows:

best_run = optim.minimize(model=create_model,
                          data=data,
                          algo=tpe.suggest,
                          max_evals=10,
                          trials=Trials())

In this example we use at most 10 evaluation runs and the TPE algorithm from hyperopt for optimization.

Check the "complete example" below for more details.

Complete example

Note: It is important to wrap your data and model into functions as shown below, and then pass them as parameters to the minimizer. data() returns the data the create_model() needs. An extended version of the above example in one script reads as follows. This example shows many potential use cases of hyperas, including:

  • Varying dropout probabilities, sampling from a uniform distribution
  • Different layer output sizes
  • Different optimization algorithms to use
  • Varying choices of activation functions
  • Conditionally adding layers depending on a choice
  • Swapping whole sets of layers
from __future__ import print_function
import numpy as np

from hyperopt import Trials, STATUS_OK, tpe
from keras.datasets import mnist
from keras.layers.core import Dense, Dropout, Activation
from keras.models import Sequential
from keras.utils import np_utils

from hyperas import optim
from hyperas.distributions import choice, uniform


def data():
    """
    Data providing function:

    This function is separated from create_model() so that hyperopt
    won't reload data for each evaluation run.
    """
    (x_train, y_train), (x_test, y_test) = mnist.load_data()
    x_train = x_train.reshape(60000, 784)
    x_test = x_test.reshape(10000, 784)
    x_train = x_train.astype('float32')
    x_test = x_test.astype('float32')
    x_train /= 255
    x_test /= 255
    nb_classes = 10
    y_train = np_utils.to_categorical(y_train, nb_classes)
    y_test = np_utils.to_categorical(y_test, nb_classes)
    return x_train, y_train, x_test, y_test


def create_model(x_train, y_train, x_test, y_test):
    """
    Model providing function:

    Create Keras model with double curly brackets dropped-in as needed.
    Return value has to be a valid python dictionary with two customary keys:
        - loss: Specify a numeric evaluation metric to be minimized
        - status: Just use STATUS_OK and see hyperopt documentation if not feasible
    The last one is optional, though recommended, namely:
        - model: specify the model just created so that we can later use it again.
    """
    model = Sequential()
    model.add(Dense(512, input_shape=(784,)))
    model.add(Activation('relu'))
    model.add(Dropout({{uniform(0, 1)}}))
    model.add(Dense({{choice([256, 512, 1024])}}))
    model.add(Activation({{choice(['relu', 'sigmoid'])}}))
    model.add(Dropout({{uniform(0, 1)}}))

    # If we choose 'four', add an additional fourth layer
    if {{choice(['three', 'four'])}} == 'four':
        model.add(Dense(100))

        # We can also choose between complete sets of layers

        model.add({{choice([Dropout(0.5), Activation('linear')])}})
        model.add(Activation('relu'))

    model.add(Dense(10))
    model.add(Activation('softmax'))

    model.compile(loss='categorical_crossentropy', metrics=['accuracy'],
                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})

    result = model.fit(x_train, y_train,
              batch_size={{choice([64, 128])}},
              epochs=2,
              verbose=2,
              validation_split=0.1)
    #get the highest validation accuracy of the training epochs
    validation_acc = np.amax(result.history['val_acc']) 
    print('Best validation acc of epoch:', validation_acc)
    return {'loss': -validation_acc, 'status': STATUS_OK, 'model': model}


if __name__ == '__main__':
    best_run, best_model = optim.minimize(model=create_model,
                                          data=data,
                                          algo=tpe.suggest,
                                          max_evals=5,
                                          trials=Trials())
    X_train, Y_train, X_test, Y_test = data()
    print("Evalutation of best performing model:")
    print(best_model.evaluate(X_test, Y_test))
    print("Best performing model chosen hyper-parameters:")
    print(best_run)

FAQ

Here is a list of a few popular errors

TypeError: require string label

You're probably trying to execute the model creation code, with the templates, directly in python. That fails simply because python cannot run the templating in the braces, e.g. {{uniform..}}. The def create_model(...) function is in fact not a valid python function anymore.

You need to wrap your code in a def create_model(...): ... function, and then call it from optim.minimize(model=create_model,... like in the example.

The reason for this is that hyperas works by doing template replacement of everything in the {{...}} into a separate temporary file, and then running the model with the replaced braces (think jinja templating).

This is the basis of how hyperas simplifies usage of hyperopt by being a "very simple wrapper".

TypeError: 'generator' object is not subscriptable

This is currently a known issue.

Just pip install networkx==1.11

NameError: global name 'X_train' is not defined

Maybe you forgot to return the x_train argument in the def create_model(x_train...) call from the def data(): ... function.

You are not restricted to the same list of arguments as in the example. Any arguments you return from data() will be passed to create_model()

notebook adjustment

If you find error like "No such file or directory" or OSError, Err22, you may need add notebook_name='simple_notebook'(assume your current notebook name is simple_notebook) in optim.minimize function like this:

best_run, best_model = optim.minimize(model=model,
                                      data=data,
                                      algo=tpe.suggest,
                                      max_evals=5,
                                      trials=Trials(),
                                      notebook_name='simple_notebook')

How does hyperas work?

All we do is parse the data and model templates and translate them into proper hyperopt by reconstructing the space object that's then passed to fmin. Most of the relevant code is found in optim.py and utils.py.

How to read the output of a hyperas model?

Hyperas translates your script into hyperopt compliant code, see here for some guidance on how to interpret the result.

How to pass arguments to data?

Suppose you want your data function take an argument, specify it like this using positional arguments only (not keyword arguments):

import pickle
def data(fname):
    with open(fname,'rb') as fh:
        return pickle.load(fh)

Note that your arguments must be implemented such that repr can show them in their entirety (such as strings and numbers). If you want more complex objects, use the passed arguments to build them inside the data function.

And when you run your trials, pass a tuple of arguments to be substituted in as data_args:

best_run, best_model = optim.minimize(
    model=model,
    data=data,
    algo=tpe.suggest,
    max_evals=64,
    trials=Trials(),
    data_args=('my_file.pkl',)
)

What if I need more flexibility loading data and adapting my model?

Hyperas is a convenience wrapper around Hyperopt that has some limitations. If it's not convenient to use in your situation, simply don't use it -- and choose Hyperopt instead. All you can do with Hyperas you can also do with Hyperopt, it's just a different way of defining your model. If you want to squeeze some flexibility out of Hyperas anyway, take a look here.

Running hyperas in parallel?

You can use hyperas to run multiple models in parallel with the use of mongodb (which you'll need to install and setup users for). Here's a short example using MNIST:

Copy and modify examples/mnist_distributed.py (bump up max_evals if you like):

Run python mnist_distributed.py. It will create a temp_model.py file. Copy this file to any machines that will be evaluating models. It will then begin waiting for evaluation results

On your other machines (make sure they have a python installed with all your dependencies, ideally with the same versions) run:

export PYTHONPATH=/path/to/temp_model.py
hyperopt-mongo-worker --exp-key='mnist_test' --mongo='mongo://username:pass@mongodb.host:27017/jobs'

Once max_evals have been completed, you should get an output with your best model. You can also look through your mongodb and examine the results, to get the best model out and run it, do:

from pymongo import MongoClient
from keras.models import load_model
import tempfile
c = MongoClient('mongodb://username:pass@mongodb.host:27017/jobs')
best_model = c['jobs']['jobs'].find_one({'exp_key': 'mnist_test'}, sort=[('result.loss', -1)])
temp_name = tempfile.gettempdir()+'/'+next(tempfile._get_candidate_names()) + '.h5'
with open(temp_name, 'wb') as outfile:
    outfile.write(best_model['result']['model_serial'])
model = load_model(temp_name)

Author: maxpumperla
Source Code: https://github.com/maxpumperla/hyperas
License: MIT License

#python #keras 

Myriam  Rogahn

Myriam Rogahn

1595216940

Keras Ecosystem - Keras Open Source Frameworks

Keras, other than being a high-level deep learning API also has some other initiatives for machine learning workflow. There is a wide range of machine learning frameworks whose development is based on Keras. In this article, we will discuss Keras Ecosystem. This ecosystem of frameworks tries to ease and optimize the current approach of training and deploying ML models.

_Keeping you updated with latest technology trends, _Join DataFlair on Telegram

Keras Ecosystem

Some of the frameworks for Keras Ecosystem are:

1. Auto Keras

This framework was built at the DATA lab with an ambition of making machine learning accessible to everyone.

It is a simple interface to perform many machine learning tasks. The supported tasks in auto Keras are image classifier, image regression, text classification, text regression, structured data classification, and structured data regression.

An example of image classification task using auto Keras:

#keras tutorials #keras ecosystem #keras frameworks #keras open source frameworks