Agnes  Sauer

Agnes Sauer

1594621980

Multi-Task Learning with Pytorch and FastAI

Following the concepts presented on my post named Should you use FastAI?, I’d like to show here how to train a Multi-Task deep learning model using the hybrid Pytorch-FastAI approach. The basic idea from the Pytorch-FastAI approach is to define a dataset and a model using Pytorch code and then use FastAI to fit your model. This approach gives you the flexibility to build complicated datasets and models but still be able to use high level FastAI functionality.

Multi-Task Learning (MTL) model is a model that is able to do more than one task. It is as simple as that. In general, as soon as you find yourself optimizing more than one loss function, you are effectively doing MTL.

Image for post

In this demonstration I’ll use the UTKFace dataset. This dataset consists of more than 30k images with labels for age, gender and ethnicity.

Image for post
If you want skip all the talking and jump to the code, here is the [

1. Why Multi-Task Learning?

When you look to someone’s picture and try to predict age, gender and ethnicity, you’re not using completely different parts of your brain right? What I’m trying to say is that you don’t try to understand the image in 3 different ways, each one specific to each task. What you’re doing is using a single understanding that your brain makes of the image and then trying to decode that understanding into age, gender and ethnicity. And besides that, there is knowledge from gender estimation that might help on age estimation, there is knowledge from ethnicity estimation that might help on gender estimation and so forth.

So, why MTL? We’re confident that training a single model to do all tasks we’re interested in yields better results than training a model for each task. Rich Caruana summarizes the goal of MTL pretty nicely: “MTL improves generalization by leveraging the domain-specific information contained in the training signals of related tasks”.

As this is a practical tutorial for training MTL models I’ll not dig deeper into theory and intuition but if you want to read more, check out this amazing post from Sebastian Ruder.

#pytorch #artificial-intelligence #deep-learning #data-science #fastai #data analysis

What is GEEK

Buddha Community

Multi-Task Learning with Pytorch and FastAI
Adaline  Kulas

Adaline Kulas

1594162500

Multi-cloud Spending: 8 Tips To Lower Cost

A multi-cloud approach is nothing but leveraging two or more cloud platforms for meeting the various business requirements of an enterprise. The multi-cloud IT environment incorporates different clouds from multiple vendors and negates the dependence on a single public cloud service provider. Thus enterprises can choose specific services from multiple public clouds and reap the benefits of each.

Given its affordability and agility, most enterprises opt for a multi-cloud approach in cloud computing now. A 2018 survey on the public cloud services market points out that 81% of the respondents use services from two or more providers. Subsequently, the cloud computing services market has reported incredible growth in recent times. The worldwide public cloud services market is all set to reach $500 billion in the next four years, according to IDC.

By choosing multi-cloud solutions strategically, enterprises can optimize the benefits of cloud computing and aim for some key competitive advantages. They can avoid the lengthy and cumbersome processes involved in buying, installing and testing high-priced systems. The IaaS and PaaS solutions have become a windfall for the enterprise’s budget as it does not incur huge up-front capital expenditure.

However, cost optimization is still a challenge while facilitating a multi-cloud environment and a large number of enterprises end up overpaying with or without realizing it. The below-mentioned tips would help you ensure the money is spent wisely on cloud computing services.

  • Deactivate underused or unattached resources

Most organizations tend to get wrong with simple things which turn out to be the root cause for needless spending and resource wastage. The first step to cost optimization in your cloud strategy is to identify underutilized resources that you have been paying for.

Enterprises often continue to pay for resources that have been purchased earlier but are no longer useful. Identifying such unused and unattached resources and deactivating it on a regular basis brings you one step closer to cost optimization. If needed, you can deploy automated cloud management tools that are largely helpful in providing the analytics needed to optimize the cloud spending and cut costs on an ongoing basis.

  • Figure out idle instances

Another key cost optimization strategy is to identify the idle computing instances and consolidate them into fewer instances. An idle computing instance may require a CPU utilization level of 1-5%, but you may be billed by the service provider for 100% for the same instance.

Every enterprise will have such non-production instances that constitute unnecessary storage space and lead to overpaying. Re-evaluating your resource allocations regularly and removing unnecessary storage may help you save money significantly. Resource allocation is not only a matter of CPU and memory but also it is linked to the storage, network, and various other factors.

  • Deploy monitoring mechanisms

The key to efficient cost reduction in cloud computing technology lies in proactive monitoring. A comprehensive view of the cloud usage helps enterprises to monitor and minimize unnecessary spending. You can make use of various mechanisms for monitoring computing demand.

For instance, you can use a heatmap to understand the highs and lows in computing visually. This heat map indicates the start and stop times which in turn lead to reduced costs. You can also deploy automated tools that help organizations to schedule instances to start and stop. By following a heatmap, you can understand whether it is safe to shut down servers on holidays or weekends.

#cloud computing services #all #hybrid cloud #cloud #multi-cloud strategy #cloud spend #multi-cloud spending #multi cloud adoption #why multi cloud #multi cloud trends #multi cloud companies #multi cloud research #multi cloud market

Agnes  Sauer

Agnes Sauer

1594621980

Multi-Task Learning with Pytorch and FastAI

Following the concepts presented on my post named Should you use FastAI?, I’d like to show here how to train a Multi-Task deep learning model using the hybrid Pytorch-FastAI approach. The basic idea from the Pytorch-FastAI approach is to define a dataset and a model using Pytorch code and then use FastAI to fit your model. This approach gives you the flexibility to build complicated datasets and models but still be able to use high level FastAI functionality.

Multi-Task Learning (MTL) model is a model that is able to do more than one task. It is as simple as that. In general, as soon as you find yourself optimizing more than one loss function, you are effectively doing MTL.

Image for post

In this demonstration I’ll use the UTKFace dataset. This dataset consists of more than 30k images with labels for age, gender and ethnicity.

Image for post
If you want skip all the talking and jump to the code, here is the [

1. Why Multi-Task Learning?

When you look to someone’s picture and try to predict age, gender and ethnicity, you’re not using completely different parts of your brain right? What I’m trying to say is that you don’t try to understand the image in 3 different ways, each one specific to each task. What you’re doing is using a single understanding that your brain makes of the image and then trying to decode that understanding into age, gender and ethnicity. And besides that, there is knowledge from gender estimation that might help on age estimation, there is knowledge from ethnicity estimation that might help on gender estimation and so forth.

So, why MTL? We’re confident that training a single model to do all tasks we’re interested in yields better results than training a model for each task. Rich Caruana summarizes the goal of MTL pretty nicely: “MTL improves generalization by leveraging the domain-specific information contained in the training signals of related tasks”.

As this is a practical tutorial for training MTL models I’ll not dig deeper into theory and intuition but if you want to read more, check out this amazing post from Sebastian Ruder.

#pytorch #artificial-intelligence #deep-learning #data-science #fastai #data analysis

PyTorch For Deep Learning 

What is Pytorch ?

Pytorch is a Deep Learning Library Devoloped by Facebook. it can be used for various purposes such as Natural Language Processing , Computer Vision, etc

Prerequisites

Python, Numpy, Pandas and Matplotlib

Tensor Basics

What is a tensor ?

A Tensor is a n-dimensional array of elements. In pytorch, everything is a defined as a tensor.

#pytorch #pytorch-tutorial #pytorch-course #deep-learning-course #deep-learning

Angela  Dickens

Angela Dickens

1598002800

Reducing computational constraints in SimCLR

In a previous blog post, we implemented the SimCLR framework in PyTorch. It was a fun exercise to understand and implement it on a simple dataset of 5 categories with a total of just 1250 training images. From the SimCLR paper, we saw how the framework benefits from larger models and larger batch sizes and can produce results comparable to those of supervised models if enough computing power is available. But these requirements make the framework quite computation-heavy. Wouldn’t it be wonderful if we could have the simplicity and power of this framework and have fewer compute requirements so that this can become accessible to everyone? Moco-v2 comes to the rescue.

Datasets

We will implement Moco-v2 in PyTorch on much bigger datasets this time and train our model on Google Colab. We will work with the Imagenette and Imagewoof datasets this time, made by Jeremy Howard from Fast.AI.

Image for post

Some images from the Imagenette datase

Image for post

Some images from the Imagewoof dataset

A quick summary of these datasets (more info is here):

  • Imagenette consists of 10 easily classified classes from Imagenet with a total of 9479 training and 3935 validation set images.
  • Imagewoof is a dataset of 10 difficult classes from Imagenet — difficult because all classes are dog breeds. There’re a total of 9035 training, and 3939 validation set images.

Contrastive Learning — A Review

The way contrastive learning works in self-supervised learning is based on the idea that we want different outlooks of images from the same category to have similar representations. But since we don’t know which images belong to the same category, what is generally done is that representations of different outlooks of the same image are brought closer to each other. These different views taken pairwise are called as positive pairs.

#fastai #self-supervised-learning #contrastive-learning #deep-learning #pytorch #deep learning

Jupyter Notebook Kernel for Running ansible Tasks and Playbooks

Ansible Jupyter Kernel

Example Jupyter Usage

The Ansible Jupyter Kernel adds a kernel backend for Jupyter to interface directly with Ansible and construct plays and tasks and execute them on the fly.

Demo

Demo

Installation:

ansible-kernel is available to be installed from pypi but you can also install it locally. The setup package itself will register the kernel with Jupyter automatically.

From pypi

pip install ansible-kernel
python -m ansible_kernel.install

From a local checkout

pip install -e .
python -m ansible_kernel.install

For Anaconda/Miniconda

pip install ansible-kernel
python -m ansible_kernel.install --sys-prefix

Usage

Local install

    jupyter notebook
    # In the notebook interface, select Ansible from the 'New' menu

Container

docker run -p 8888:8888 benthomasson/ansible-jupyter-kernel

Then copy the URL from the output into your browser:
http://localhost:8888/?token=ABCD1234

Using the Cells

Normally Ansible brings together various components in different files and locations to launch a playbook and performs automation tasks. For this jupyter interface you need to provide this information in cells by denoting what the cell contains and then finally writing your tasks that will make use of them. There are Examples available to help you, in this section we'll go over the currently supported cell types.

In order to denote what the cell contains you should prefix it with a pound/hash symbol (#) and the type as listed here as the first line as shown in the examples below.

#inventory

The inventory that your tasks will use

#inventory
[all]
ahost ansible_connection=local
anotherhost examplevar=val

#play

This represents the opening block of a typical Ansible play

#play
name: Hello World
hosts: all
gather_facts: false

#task

This is the default cell type if no type is given for the first line

#task
debug:
#task
shell: cat /tmp/afile
register: output

#host_vars

This takes an argument that represents the hostname. Variables defined in this file will be available in the tasks for that host.

#host_vars Host1
hostname: host1

#group_vars

This takes an argument that represents the group name. Variables defined in this file will be available in the tasks for hosts in that group.

#group_vars BranchOfficeX
gateway: 192.168.1.254

#vars

This takes an argument that represents the filename for use in later cells

#vars example_vars
message: hello vars
#play
name: hello world
hosts: localhost
gather_facts: false
vars_files:
    - example_vars

#template

This takes an argument in order to create a templated file that can be used in later cells

#template hello.j2
{{ message }}
#task
template:
    src: hello.j2
    dest: /tmp/hello

#ansible.cfg

Provides overrides typically found in ansible.cfg

#ansible.cfg
[defaults]
host_key_checking=False

Examples

You can find various example notebooks in the repository

Using the development environment

It's possible to use whatever python development process you feel comfortable with. The repository itself includes mechanisms for using pipenv

pipenv install
...
pipenv shell

Author: ansible
Source Code:  https://github.com/ansible/ansible-jupyter-kernel
License: Apache-2.0 License

#jupyter #python