Pytorch Tutorial : PyTorch Basics for Beginners

The world is changing and so is the technology serving it. It’s crucial for everyone to keep up with the rapid changes in technology. One of the domains which is witnessing the fastest and largest evolution is Artificial Intelligence.

We are training our machines to learn and the results are now getting better and better. There are GANs which can generate new images, Deep Learning models for translating signed language into text, and what not! In this swift-moving domain, PyTorch has originated as a new choice for building these models.

PyTorch is a Python-based library which facilitates building Deep Learning models and using them in various applications. But, it’s more than just another Deep Learning library, it’s a scientific computing package (as the official PyTorch docs state).

In this piece about Pytorch Tutorial, I talk about the new platform in Deep Learning.

The latest version of the platform brings a lot of new capabilities to the table and is clocking vibrant support from the whole industry. It is remarkable how Pytorch is being touted as a serious contender to Google’s Tensorflow just within a couple of years of its release. Its popularity is mainly being driven by a smoother learning curve and a cleaner interface, which is providing developers with a more intuitive approach to build neural networks.

So what’s new in Pytorch 1.0? Here are the highlights of the new release:

Torch Script

PyTorch 1.0 introduces JIT for model graphs that revolve around the concept of Torch Script which is a restricted subset of the Python language. It has its very own compiler and transform passes, optimizations, etc. Class and method annotations are used to indicate the scripts as a part of the Python code. Examples include @torch.jit.script, and @torch.jit.script_method. Annotations help to preserve elements such as loops, print statements, and control flow.

• Although it should be noted that you need to remove the annotations in the case you want to:

• Debug the scripts using standard Python tools

Switch to eager execution modeSubsets that are valid Torch Scripts include:

• Tensors and numeric primitives

• If statements

• Simple Loops

• Code organizations with nn.Module

• Tuples, lists, strings, and print

• Gradient propagation through script functions

• In-place updates to tensors and lists

• Direct use of standard nn.Modules such as nn/ConvCalling functions like grad() or backwards() within @scripts
  You can work with Torch Scripts in two ways:

• Tracing Mode

• Scripting Mode

1. Tracing Mode

The Pytorch tracer (torch.jit.trace) records native Pytorch operations that are executed in a code region. Along with this, it also records the data dependencies between them. Although it has had a tracer since version 0.3, it can now re-execute the tracer for you by leveraging a high-performance C++ runtime environment. The trace no longer needs to be executed elsewhere as the latest version has made it possible to integrate the optimizations and hardware integrations of Caffe2.You can also create Torch Scripts by using a tracing JIT. Here the computational graph nodes will be visited and the final script will be produced after recording the operations.

2. Scripting Mode

The Scripting Mode makes it possible for you to write regular Python functions without the need to use complicated language functions. The @script decorator can be used to compile a function once the desired functionality has been isolated.Such an annotation would directly transform the Python function into a C++ runtime for higher performance.Torch Scripts can be created by providing custom scripts where you provide the description of your model. Though it is necessary to take the limitations of Torch Scripts into account for this purpose.

3. Integration of Research and Production

PyTorch 1.0 integrates research with production very intuitively. Although past versions quickly rose to popularity for the flexibility they provided in Artificial Intelligence development and research, performance at production scale remained a challenge. Developers had to translate the research code into a graph model representation in Caffe2 for production purposes. Such a migration used to be manual and time-consuming.But now PyTorch 1.0 integrates immediate and graph execution modes to help developers handle research and production simultaneously. With the help of a hybrid front-end, you can now share code between both the modes for seamless prototyping and production.

4. C++ API and Frontend

Python has not been a popular option for deployment in C++ due to factors such as high overheads on small models, multi-threading services bottleneck on GIL, etc. PyTorch 1.0 provides developers with a two-way pathway from Python to C++ and vice versa. This helps in functions such as debugging and refactoring. The C++ API allows you to write custom implementations such as calls to third-party functions.In addition to this, the beta version of C++ Frontend was also announced. Though it is currently marked as ‘API Unstable’. This makes it ready to be used for building research applications but its utilization for production purposes will take some time to stabilize.

5. Optimization and Export

It does not matter whether you are using the Tracing mode or the Scripting mode. With PyTorch 1.0, the result is always a Python free representation of your model which can be used in two ways - to optimize the model or export the model - in the production environments.
Whole program optimizations become possible with the ability to extract bigger segments of the model into an intermediate representation. Computations can also be offloaded to specialized AI accelerators. PyTorch 1.0 also includes passes to fuse GPU operations together and improve the performance of smaller RNN models.

Support From the Ecosystem

The tech world has been quick to respond to the added capabilities of PyTorch with major market players announcing extended support to create a thriving ecosystem around the Deep Learning platform. Here is a wrap up of the major announcements that the release of PyTorch 1.0 has attracted:

• Amazon Web Services (AWS):Amazon SageMaker now features PyTorch 1.0 images. Developers can even train and deploy their Deep Learning models of PyTorch in SageMarker. Once the PyTorch script has been written, Amazon SageMaker training can handle subsequent tasks such as setting up the distributed training cluster, hyperparameter tuning and transferring of data. This makes Amazon SageMaker as a managed online endpoint with a coveted ability to scale up automatically whenever the need arises.Using PyTorch in Amazon SageMarker starts with the developer providing the relevant script and then use the PyTorch estimator from Amazon SageMarker Python SDK as follows:

estimator = PyTorch(entry_point="pytorch_script.py",
role=role,
train_instance_count=2,
train_instance_type='ml.p2.xlarge',
hyperparameters={'epochs': 10,
'lr': 0.01})

Developers can package their code as a Docker container to host it or deploy it for inference.

• Microsoft:
  Microsoft has also been quick in announcing major support for PyTorch. The highlights include:Setting up extensive Windows support for PyTorch. Actively contributing to the GitHub code.Allocating a dedicated team of Developers to improve PyTorch.Closely working with the community.Integration of PyTorch in all Machine Learning products of Microsoft which includes:

• VS Code

• Azure

• Data Science VM

• Azure ML

• Google Cloud Platform (GCP):
  Google has not held back in any way by jumping into the mix with a few major announcements of its own in partnership with the PyTorch 1.0 release:Although Kubeflow already supported PyTorch, Google has extended the TensorRT package in Kubeflow to support serving PyTorch models.A collaboration of Tenserboard with PyTorch. This includes Cloud TPU and TPU pods with support for easy scaling.Broadened support for PyTorch throughout the AI platforms and services of Google Cloud Support.Fully hybrid Python and C/C++ front-end support and native distribution execution support for production environments.

• Nvidia:
  Nvidia and Facebook also have a healthy collaboration history with the companies joining hands together in 2017 to create large-scale distributed training scenarios to develop Machine Learning based applications for Edge devices. Collaborative efforts continue today with Nvidia actively working to integrate Pytorch into their current offerings:

• A PyTorch Extension Tools (APEX) for easy Mixed Precision and Distributed Training.

• Support for PyTorch framework across the inference workflow. Developers can:

• Import PyTorch models with the ONNX format

• Apply INT8 and FP16 optimizations

• Calibrate for lower precision with high accuracy

• Generate runtimes for production deployment

• Availability of PyTorch container from the Nvidia GPU Cloud container registry to help developers get started quickly with the platform.

Key Capabilities and Features

Having started out just 2 years ago, it is incredible how quickly it has matured to add new capabilities and functionalities. A host of improved abilities have been introduced in PyTorch 1.0. Here is a quick look at what the open source Deep Learning platform is capable of today:

• Hybrid Front-end: Provides ease of use and better flexibility in eager mode. Provides graph mode for speed, optimization, and functionality in C++ runtime environments.

• Distributed Training: Optimized performance for both research and production. Provides asynchronous execution of collective operations and peer to peer communication.

• Python First: PyTorch has been built to be deeply integrated with Python and can be actively used with popular libraries and packages such as Cython and Numba.

• Tools and Libraries: The community of PyTorch is highly active, which has led to the development of a rich ecosystem of tools and libraries. This has extended the reach and supported development in numerous areas.

• Native ONNX Support: PyTorch also offers export models in the standard Open Neural Network Exchange format. This provides developers with direct access to ONNX-compatible platforms, runtimes, visualizers, etc.

• C++ Front-end: A C++ interface that is intended to enable research in high performance or low latency C++ applications.

• Cloud Partners: As established by the support provided from the ecosystem, all major cloud computing platforms are supporting PyTorch today. This paves way for a smooth development process, easy scaling, large-scale training on GPUs, etc.

Main Elements in PyTorch

If you are planning to fuel your development process by leveraging the phenomenal capabilities, there are some main elements that you should know about before starting out to plan your development process in the most optimum way. Let’s take a look:

1. PyTorch Tensors

Tensors are multidimensional arrays. Tensors are similar to numpy’s ndarrays, though they can also be used on GPUs. A simple one-dimensional matrix can be defined as:

# import pytorch
//import torch
# define a tensor
torch.FloatTensor([2])

2[torch.FloatTensor of size 1]

2. Mathematical Operations
PyTorch provides you with 200+ mathematical operators to work with. This meets the need of a scientific computing library making efficient implementations of mathematical functions. Here is how addition works out :

a = torch.FloatTensor([2])
b = torch.FloatTensor([1])
a + b

3[torch.FloatTensor of size 1]

Various functions on matrices can also be performed on the defined PyTorch Tensors. Here’s an example:

matrix = torch.randn(3, 3)
matrix
0.4182 2.1159 8.3576
-0.4563 -0.2357 -2.5800
-0.5081 -2.1937 -0.0291
[torch.FloatTensor of size 3x3]

matrix.t()

0.4182 -0.4563 -0.5081
2.1159 -0.2357 -2.1937
8.3576 -2.5800 -0.0291
[torch.FloatTensor of size 3x3]

3. Autograd Module

PyTorch makes use of Automatic differentiation. A recorder records all the performed operations and then plays it back to compute the gradients. This technique finds extensive usage when neural networks are built.from torch.autograd import Variable

x = Variable(train_x)
y = Variable(train_y, requires_grad=False)

• Optim Module

The torch.optim module helps you to implement optimization algorithms to build neural networks. The best feature is the support for most of the commonly used methods. This eliminates the need to build them from scratch.
For instance, here is how you can use the Adam.optimizer:

optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

• nn Module

It can be difficult to define complex neural networks with raw autograd. The nn Module helps in this regard by allowing you to define a set of modules that can be considered as a neural network layer.

import torch
# define model
model = torch.nn.Sequential(
torch.nn.Linear(input_num_units, hidden_num_units),
torch.nn.ReLU(),
torch.nn.Linear(hidden_num_units, output_num_units),
)
loss_fn = torch.nn.CrossEntropyLoss()

Important Things to Keep in Mind

With the basics covered, you can now kickstart building your very own neural network with PyTorch and make use of the maturing ecosystem to bring your ideas to life.

But before you begin, here are some important details to keep in mind to avoid certain pitfalls that might give you trouble at a later stage.

1. Data Types
Data types matter a lot in PyTorch. For example, not all NumPy arrays can be converted to torch Tensor. Only certain NumPy data types can be converted to torch Tensor type such as numpy.uint8 to torch.ByteTensor, numpy.int16 to torch.ShortTensor, and numpy.int32 to torch.IntTensor.

2. Numerical Stability

The thumb rule is - if it can overflow or underflow, it probably will. For instance, let’s say that you have to relate samples with tags in both positive and negative ways. You use the classical sigmoid + log loss for this purpose.

sigmoid = torch.nn.functional.sigmoiddot_p = torch.dot(anchor, tag_p)

loss_pos = -torch.log(sigmoid(dot_p)) #(1)

dot_n = torch.dot(anchor, tag_n)

loss_neg = -torch.log(1 - sigmoid(dot_n)) #(2)

Log(0) is the critical point here. Since Log is undefined for this input, there are two ways in which this situation can go down:

sigmoid(x) = 0, which means x is a “large” negative value.
sigmoid(x) = 1, which means x is a “large” positive value.

Regardless of the case, -log(y) evaluates to zero. This leads to a numerical instability which hinders with further optimization steps.
A workaround here can be to bound the values of sigmoid to be slightly below one and slightly above zero.

value = torch.nn.functional.sigmoid(x)

value = torch.clamp(torch.clamp(value, min=eps), max=1-eps)

This makes sigmoid(dot_p) to be always positive and (1 – sigmoid(dot_n)) to never amount to zero. Although this is not rocket science, you need to keep such evaluations in mind to ensure numerical stability while you code.

3. Gradients
In PyTorch, Gradients accumulate by default. To understand this, consider a scenario in which you run a computation once, both forward and backward, and everything seems to be working correctly. But when you run it for the second time, new gradients get added to the gradients from the first operation. This is easy to forget, especially for developers who are dealing with a Machine Learning platform/library for the first time.A quick solution in such a scenario would be to manually set the gradients to zero between every two runs. This can be done with:

w.grad.data.zero()

Conclusion

PyTorch is taking the world of Deep Learning by storm by paving way for better innovation in the whole ecosystem that even includes the likes of education providers such as Udacity and Fast.ai. All this and more makes the future of PyTorch quite promising and provides huge incentives to developers to start depending on the platform confidently. Subscribe to the blog for further tutorials and updates on PyTorch.

#python

What is GEEK

Buddha Community

Data Visualization in R with ggplot2: A Beginner Tutorial

A famous general is thought to have said, “A good sketch is better than a long speech.” That advice may have come from the battlefield, but it’s applicable in lots of other areas — including data science. “Sketching” out our data by visualizing it using ggplot2 in R is more impactful than simply describing the trends we find.

This is why we visualize data. We visualize data because it’s easier to learn from something that we can see rather than read. And thankfully for data analysts and data scientists who use R, there’s a tidyverse package called ggplot2 that makes data visualization a snap!

In this blog post, we’ll learn how to take some data and produce a visualization using R. To work through it, it’s best if you already have an understanding of R programming syntax, but you don’t need to be an expert or have any prior experience working with ggplot2

#data science tutorials #beginner #ggplot2 #r #r tutorial #r tutorials #rstats #tutorial #tutorials

Tutorial: Getting Started with R and RStudio

In this tutorial we’ll learn how to begin programming with R using RStudio. We’ll install R, and RStudio RStudio, an extremely popular development environment for R. We’ll learn the key RStudio features in order to start programming in R on our own.

If you already know how to use RStudio and want to learn some tips, tricks, and shortcuts, check out this Dataquest blog post.

Table of Contents

• 1. Install R
• 2. Install RStudio
• 3. First Look at RStudio
• 4. The Console
• 5. The Global Environment
• 6. Install the [tidyverse](https://www.dataquest.io/blog/tutorial-getting-started-with-r-and-rstudio/#tve-jump-173bb26184b) Packages
• 7. Load the [tidyverse](https://www.dataquest.io/blog/tutorial-getting-started-with-r-and-rstudio/#tve-jump-173bb264c2b) Packages into Memory
• 8. Identify Loaded Packages
• 9. Get Help on a Package
• 10. Get Help on a Function
• 11. RStudio Projects
• 12. Save Your “Real” Work. Delete the Rest.
• 13. R Scripts
• 14. Run Code
• 15. Access Built-in Datasets
• 16. Style
• 17. Reproducible Reports with R Markdown
• 18. Use RStudio Cloud
• 19. Get Your Hands Dirty!
• Additional Resources
• Bonus: Cheatsheets

#data science tutorials #beginner #r tutorial #r tutorials #rstats #tutorial #tutorials

Tutorial: Loading and Cleaning Data with R and the tidyverse

1. Characteristics of Clean Data and Messy Data

What exactly is clean data? Clean data is accurate, complete, and in a format that is ready to analyze. Characteristics of clean data include data that are:

• Free of duplicate rows/values
• Error-free (e.g. free of misspellings)
• Relevant (e.g. free of special characters)
• The appropriate data type for analysis
• Free of outliers (or only contain outliers have been identified/understood), and
• Follows a “tidy data” structure

Common symptoms of messy data include data that contain:

• Special characters (e.g. commas in numeric values)
• Numeric values stored as text/character data types
• Duplicate rows
• Misspellings
• Inaccuracies
• White space
• Missing data
• Zeros instead of null values

2. Motivation

In this blog post, we will work with five property-sales datasets that are publicly available on the New York City Department of Finance Rolling Sales Data website. We encourage you to download the datasets and follow along! Each file contains one year of real estate sales data for one of New York City’s five boroughs. We will work with the following Microsoft Excel files:

• rollingsales_bronx.xls
• rollingsales_brooklyn.xls
• rollingsales_manhattan.xls
• rollingsales_queens.xls
• rollingsales_statenisland.xls

As we work through this blog post, imagine that you are helping a friend launch their home-inspection business in New York City. You offer to help them by analyzing the data to better understand the real-estate market. But you realize that before you can analyze the data in R, you will need to diagnose and clean it first. And before you can diagnose the data, you will need to load it into R!

3. Load Data into R with readxl

Benefits of using tidyverse tools are often evident in the data-loading process. In many cases, the tidyverse package readxl will clean some data for you as Microsoft Excel data is loaded into R. If you are working with CSV data, the tidyverse readr package function read_csv() is the function to use (we’ll cover that later).

Let’s look at an example. Here’s how the Excel file for the Brooklyn borough looks:

The Brooklyn Excel file

Now let’s load the Brooklyn dataset into R from an Excel file. We’ll use the readxlpackage. We specify the function argument skip = 4 because the row that we want to use as the header (i.e. column names) is actually row 5. We can ignore the first four rows entirely and load the data into R beginning at row 5. Here’s the code:

library(readxl) # Load Excel files
brooklyn <- read_excel("rollingsales_brooklyn.xls", skip = 4)

Note we saved this dataset with the variable name brooklyn for future use.

4. View the Data with tidyr::glimpse()

The tidyverse offers a user-friendly way to view this data with the glimpse() function that is part of the tibble package. To use this package, we will need to load it for use in our current session. But rather than loading this package alone, we can load many of the tidyverse packages at one time. If you do not have the tidyverse collection of packages, install it on your machine using the following command in your R or R Studio session:

install.packages("tidyverse")

Once the package is installed, load it to memory:

library(tidyverse)

Now that tidyverse is loaded into memory, take a “glimpse” of the Brooklyn dataset:

glimpse(brooklyn)
## Observations: 20,185
## Variables: 21
## $ BOROUGH <chr> "3", "3", "3", "3", "3", "3", "…
## $ NEIGHBORHOOD <chr> "BATH BEACH", "BATH BEACH", "BA…
## $ `BUILDING CLASS CATEGORY` <chr> "01 ONE FAMILY DWELLINGS", "01 …
## $ `TAX CLASS AT PRESENT` <chr> "1", "1", "1", "1", "1", "1", "…
## $ BLOCK <dbl> 6359, 6360, 6364, 6367, 6371, 6…
## $ LOT <dbl> 70, 48, 74, 24, 19, 32, 65, 20,…
## $ `EASE-MENT` <lgl> NA, NA, NA, NA, NA, NA, NA, NA,…
## $ `BUILDING CLASS AT PRESENT` <chr> "S1", "A5", "A5", "A9", "A9", "…
## $ ADDRESS <chr> "8684 15TH AVENUE", "14 BAY 10T…
## $ `APARTMENT NUMBER` <chr> NA, NA, NA, NA, NA, NA, NA, NA,…
## $ `ZIP CODE` <dbl> 11228, 11228, 11214, 11214, 112…
## $ `RESIDENTIAL UNITS` <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1…
## $ `COMMERCIAL UNITS` <dbl> 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `TOTAL UNITS` <dbl> 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1…
## $ `LAND SQUARE FEET` <dbl> 1933, 2513, 2492, 1571, 2320, 3…
## $ `GROSS SQUARE FEET` <dbl> 4080, 1428, 972, 1456, 1566, 22…
## $ `YEAR BUILT` <dbl> 1930, 1930, 1950, 1935, 1930, 1…
## $ `TAX CLASS AT TIME OF SALE` <chr> "1", "1", "1", "1", "1", "1", "…
## $ `BUILDING CLASS AT TIME OF SALE` <chr> "S1", "A5", "A5", "A9", "A9", "…
## $ `SALE PRICE` <dbl> 1300000, 849000, 0, 830000, 0, …
## $ `SALE DATE` <dttm> 2020-04-28, 2020-03-18, 2019-0…

The glimpse() function provides a user-friendly way to view the column names and data types for all columns, or variables, in the data frame. With this function, we are also able to view the first few observations in the data frame. This data frame has 20,185 observations, or property sales records. And there are 21 variables, or columns.

#data science tutorials #beginner #r #r tutorial #r tutorials #rstats #tidyverse #tutorial #tutorials

Getting Started with R Markdown — Guide and Cheatsheet

In this blog post, we’ll look at how to use R Markdown. By the end, you’ll have the skills you need to produce a document or presentation using R Mardown, from scratch!

We’ll show you how to convert the default R Markdown document into a useful reference guide of your own. We encourage you to follow along by building out your own R Markdown guide, but if you prefer to just read along, that works, too!

R Markdown is an open-source tool for producing reproducible reports in R. It enables you to keep all of your code, results, plots, and writing in one place. R Markdown is particularly useful when you are producing a document for an audience that is interested in the results from your analysis, but not your code.

R Markdown is powerful because it can be used for data analysis and data science, collaborating with others, and communicating results to decision makers. With R Markdown, you have the option to export your work to numerous formats including PDF, Microsoft Word, a slideshow, or an HTML document for use in a website.

Turn your data analysis into pretty documents with R Markdown.

We’ll use the RStudio integrated development environment (IDE) to produce our R Markdown reference guide. If you’d like to learn more about RStudio, check out our list of 23 awesome RStudio tips and tricks!

Here at Dataquest, we love using R Markdown for coding in R and authoring content. In fact, we wrote this blog post in R Markdown! Also, learners on the Dataquest platform use R Markdown for completing their R projects.

We included fully-reproducible code examples in this blog post. When you’ve mastered the content in this post, check out our other blog post on R Markdown tips, tricks, and shortcuts.

Okay, let’s get started with building our very own R Markdown reference document!

R Markdown Guide and Cheatsheet: Quick Navigation

• 1. Install R Markdown
• 2. Default Output Format
• 3. R Markdown Document Format
• 4. Section Headers
• 5. Bulleted and Numbered Lists
• 6. Text Formatting
• 7. Links
• 8. Code Chunks
• 9. Running Code
• 10. Control Behavior with Code Chunk Options
• 11. Inline Code
• 12. Navigating Sections and Code Chunks
• 13. Table Formatting
• 14. Output Format Options
• 15. Presentations
• 16. Add a Table of Contents
• 17. Reproducible Reports with RStudio Cloud
• More R Markdown Tips, Tricks, and Shortcuts
• Bonus: R Markdown Cheatsheet
• Additional Resources

1. Install R Markdown

R Markdown is a free, open source tool that is installed like any other R package. Use the following command to install R Markdown:

install.packages("rmarkdown")

Now that R Markdown is installed, open a new R Markdown file in RStudio by navigating to File > New File > R Markdown…. R Markdown files have the file extension “.Rmd”.

2. Default Output Format

When you open a new R Markdown file in RStudio, a pop-up window appears that prompts you to select output format to use for the document.

The default output format is HTML. With HTML, you can easily view it in a web browser.

We recommend selecting the default HTML setting for now — it can save you time! Why? Because compiling an HTML document is generally faster than generating a PDF or other format. When you near a finished product, you change the output to the format of your choosing and then make the final touches.

One final thing to note is that the title you give your document in the pop-up above is not the file name! Navigate to File > Save As.. to name, and save, the document.

#data science tutorials #beginner #r #r markdown #r tutorial #r tutorials #rstats #rstudio #tutorial #tutorials

Laravel 8 Tutorial for Beginners

Hello everyone! I just updated this tutorial for Laravel 8. In this tutorial, we’ll go through the basics of the Laravel framework by building a simple blogging system. Note that this tutorial is only for beginners who are interested in web development but don’t know where to start. Check it out if you are interested: Laravel Tutorial For Beginners

Laravel is a very powerful framework that follows the MVC structure. It is designed for web developers who need a simple, elegant yet powerful toolkit to build a fully-featured website.

#laravel 8 tutorial #laravel 8 tutorial crud #laravel 8 tutorial point #laravel 8 auth tutorial #laravel 8 project example #laravel 8 tutorial for beginners