Nabunya  Jane

Nabunya Jane

1622519760

My PC vs Azure ML Compute — GO!

A few weeks ago, I created a YouTube video on connecting Microsoft Visual Studio Code to a Jupyter Notebook running on a Compute resource within Azure (via Azure ML Studio). When I was making the video, I kept wondering how much faster that remote server really is.

So I decided to test it.

Here’s the video for the Jupyter setup:

The Setup

I’m not going to make you wade through my exploratory data analysis, or even loading my data. That’s not germane to my purpose, which is just to show you the difference between running a significant piece of code on my personal computer and comparing it to Compute on Azure.

If you’re following along, as I stated in the Jupyter video, you also need an Azure account with an Azure ML Studio resource created. Here’s a video to show you how to do all that.

The Code

The code is running a GridSearchCV, which not only runs cross-validation but also does some relatively significant hyper-parameter tuning. I chose this scenario because it takes some time to run on my PC and represents a common situation for remote compute.

You can see the code is going through a number of KNeighborsRegressor options through 5 cross-folds. It ends up being 360 fits in total. It’s not difficult to alter this code for any estimator / tuning / CV option you want.

I also added my own timer, just to get a full elapsed time for the entire code.

My PC Results

My PC is a Microsoft Surface with an i7–8650U CPU and 16 GB of RAM. I’m running Windows 10. It’s generally pretty quick even without a GPU.

To run all 360 fits, my PC took a total of 4,287 seconds, which is 71.4 minutes or an hour, 11 minutes and about 27 seconds. Not horrible, but what’s a bit more telling is what happened to my PC over that time span.

#python #data-science #azure

What is GEEK

Buddha Community

My PC vs Azure ML Compute — GO!
Jolie  Reichert

Jolie Reichert

1596948420

How to Use Azure Go SDK to Manage Azure Data Explorer Clusters

Getting started with Azure Data Explorer using the Go SDK covered how to use the Azure Data Explorer Go SDK to ingest and query data from azure data explorer to ingest and query data. In this blog you will the Azure Go SDK to manage Azure Data Explorer clusters and databases.

Azure Data Explorer (also known as Kusto) is a fast and scalable data exploration service for analyzing large volumes of diverse data from any data source, such as websites, applications, IoT devices, and more. This data can then be used for diagnostics, monitoring, reporting, machine learning, and additional analytics capabilities.

In case you’re wondering, we are talking about two different SDKs here. The one covered in this blog is for resource administration (also known as the control plane SDK) and the the one I used in the other post is data plane SDK for interacting with the Azure Data Explorer service itself (ingestion, query etc.)

What’s Covered?

A simple CLI application is used as an example to demonstrate how to use the Go SDK. We’ll try out the application first and go through how to:

  • Create and list Azure Data Explorer clusters
  • Create and list databases in that cluster
  • Delete the database and cluster

Once that’s done, we’ll walk through the sample code to understand what’s going on

The code is available on GitHub https://github.com/abhirockzz/azure-go-sdk-for-dataexplorer

Please note that this CLI based example is just meant to showcase how to use the Azure Go SDK (in the context of Azure Data Explorer) as a part of a larger application. It is not supposed to replace/substitute the Azure CLI which can be used to manage Azure Data Explorer resources

Pre-requisites

Install Go 1.13 or above

You will need a Microsoft Azure account. Go ahead and sign up for a free one!

Install the Azure CLI if you don’t have it already (should be quick!)

#tutorial #big data #azure #analytics #go #golang #azure data explorer clusters #azure go sdk

Rylan  Becker

Rylan Becker

1620513960

AWS v/s Google v/s Azure: Who will win the Cloud War?

In the midst of this pandemic, what is allowing us unprecedented flexibility in making faster technological advancements is the availability of various competent cloud computing systems. From delivering on-demand computing services for applications, processing and storage, now is the time to make the best use of public cloud providers. What’s more, with easy scalability there are no geographical restrictions either.

Machine Learning systems can be indefinitely supported by them as they are open-sourced and within reach now more than ever with increased affordability for businesses. In fact, public cloud providers are increasingly helpful in building Machine Learning models. So, the question that arises for us is – what are the possibilities for using them for deployment as well?

What do we mean by deployment?

Model building is very much like the process of designing any product. From ideation and data preparation to prototyping and testing. Deployment basically is the actionable point of the whole process, which means that we use the already trained model and make its predictions available to users or other systems in an automated, reproducible and auditable manner.

#cyber security #aws vs azure #google vs aws #google vs azure #google vs azure vs aws

Fannie  Zemlak

Fannie Zemlak

1599854400

What's new in the go 1.15

Go announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved allocation for small objects at high core counts, X.509 CommonName deprecation, GOPROXY supports skipping proxies that return errors, New embedded tzdata package, Several Core Library improvements and more.

As Go promise for maintaining backward compatibility. After upgrading to the latest Go 1.15 version, almost all existing Golang applications or programs continue to compile and run as older Golang version.

#go #golang #go 1.15 #go features #go improvement #go package #go new features

Eric  Bukenya

Eric Bukenya

1624713540

Learn NoSQL in Azure: Diving Deeper into Azure Cosmos DB

This article is a part of the series – Learn NoSQL in Azure where we explore Azure Cosmos DB as a part of the non-relational database system used widely for a variety of applications. Azure Cosmos DB is a part of Microsoft’s serverless databases on Azure which is highly scalable and distributed across all locations that run on Azure. It is offered as a platform as a service (PAAS) from Azure and you can develop databases that have a very high throughput and very low latency. Using Azure Cosmos DB, customers can replicate their data across multiple locations across the globe and also across multiple locations within the same region. This makes Cosmos DB a highly available database service with almost 99.999% availability for reads and writes for multi-region modes and almost 99.99% availability for single-region modes.

In this article, we will focus more on how Azure Cosmos DB works behind the scenes and how can you get started with it using the Azure Portal. We will also explore how Cosmos DB is priced and understand the pricing model in detail.

How Azure Cosmos DB works

As already mentioned, Azure Cosmos DB is a multi-modal NoSQL database service that is geographically distributed across multiple Azure locations. This helps customers to deploy the databases across multiple locations around the globe. This is beneficial as it helps to reduce the read latency when the users use the application.

As you can see in the figure above, Azure Cosmos DB is distributed across the globe. Let’s suppose you have a web application that is hosted in India. In that case, the NoSQL database in India will be considered as the master database for writes and all the other databases can be considered as a read replicas. Whenever new data is generated, it is written to the database in India first and then it is synchronized with the other databases.

Consistency Levels

While maintaining data over multiple regions, the most common challenge is the latency as when the data is made available to the other databases. For example, when data is written to the database in India, users from India will be able to see that data sooner than users from the US. This is due to the latency in synchronization between the two regions. In order to overcome this, there are a few modes that customers can choose from and define how often or how soon they want their data to be made available in the other regions. Azure Cosmos DB offers five levels of consistency which are as follows:

  • Strong
  • Bounded staleness
  • Session
  • Consistent prefix
  • Eventual

In most common NoSQL databases, there are only two levels – Strong and EventualStrong being the most consistent level while Eventual is the least. However, as we move from Strong to Eventual, consistency decreases but availability and throughput increase. This is a trade-off that customers need to decide based on the criticality of their applications. If you want to read in more detail about the consistency levels, the official guide from Microsoft is the easiest to understand. You can refer to it here.

Azure Cosmos DB Pricing Model

Now that we have some idea about working with the NoSQL database – Azure Cosmos DB on Azure, let us try to understand how the database is priced. In order to work with any cloud-based services, it is essential that you have a sound knowledge of how the services are charged, otherwise, you might end up paying something much higher than your expectations.

If you browse to the pricing page of Azure Cosmos DB, you can see that there are two modes in which the database services are billed.

  • Database Operations – Whenever you execute or run queries against your NoSQL database, there are some resources being used. Azure terms these usages in terms of Request Units or RU. The amount of RU consumed per second is aggregated and billed
  • Consumed Storage – As you start storing data in your database, it will take up some space in order to store that data. This storage is billed per the standard SSD-based storage across any Azure locations globally

Let’s learn about this in more detail.

#azure #azure cosmos db #nosql #azure #nosql in azure #azure cosmos db

Overview Of Azure ML And ML Studio

In this article, we are going to learn about Azure ML and ML Studio. As we know Azure is Microsoft Cloud computing service. And Machine learning supported by Azure is called Azure ML. It’s a complete automated framework to build, teach, train and deploy as a web service and have visual development environment to make it easy for data scientists.

Benefits of having Azure ML as a cloud solution,

  1. It’s providing Azure ML Studio which it uses to create model and deploy instantly.
  2. Visual user interface, drag and drop feature, real time data visualization.
  3. All projects, experiments are stored in the cloud. You can access it from anywhere.
  4. Almost all input data types are supported by Azure ML as data source.
  5. Extend model using R and Python or use trained model as module.

Supported Input data types

  1. Hive
  2. HTTP
  3. MySQL
  4. SQL Server
  5. PostgreSQL
  6. Power Query
  7. SharePoint
  8. Azure DB
  9. Web API
  10. Local Files
  11. Teradata

There are many other data sources; you can check it on the Microsoft portal.

Azure ML Studio

Here I will give an overview of Azure ML Studio. You need to sign up in Azure portal and select Machine Learning Studio to launch it. It opens in the browser and looks like the below image.

Overview Of Azure ML And ML Studio

It’s workbench software which has predefined protocols to follow while building and training a model. As per the image, the visual workspace enables developers to quickly create models and visualize data with just some clicks.

It has 6 high level navigations menus and those are Projects, Experiments, Web Service, Dataset, Trained Model and Settings.

Overview Of Azure ML And ML Studio

Projects

It lists all projects and models created by users. Project contains combinations of all module experiments and datasets.

Experiments

It allows developers to build, test and iterate multiple times on either its new model or existing model. You can copy models and do many experiments and get accurate predictive results.

Web Service

Tested and trained models are deployed as web services as public APIs to use outside of the Azure environment. It predict results based on input parameters. It returns value based on trained deployed model data.

Dataset

Dataset contains uploaded datasets in Azure ML studio. It lists uploaded datasets and you can also pick from Microsoft sample datasets, which can be utilized for your experiments. You can use big New + buttons to add data files from your local computer.

Trained Model

Save your trained models and experiment for future uses.

Settings

Settings tab allows us to view and edit workspace and regenerate authorization token.

Overview Of Azure ML And ML Studio

#azure #overriew #azure-ml #ml-studio