Precomputation: The Key to High-Definition Cloud Analytics

From the very beginning of the big data era, large data volumes have been a source of fear, uncertainty, and doubt for those tasked with analytics: In order to work with big data, goes the argument you must make it small. Given the primitive processing tools and programming models of the time, it became obvious that no one could practically deal with “high-definition” data.

For many important algorithms — recommendations, predictions, behavior modeling — the need for a single source of truth has been supplanted by the highest probability based on the available datasets. Given the business upside of using machine learning to enable new revenue streams or greater sales pipelines, we’ve been willing to wait hours, days, or weeks for these models to play out.

But the rise of machine learning has not blunted our desire to work interactively with data — to probe and experiment with live, high-definition data. Not only is this now possible, it is also practical with the intelligent, systematic use of precomputation techniques developed as part of the open source project called Apache Kylin.

Apache Kylin is an open source project that implements the precomputation pattern in big data environments. The company behind Apache Kylin, Kyligence, was founded by the team that created the Kylin project. Kyligence provides the commercial version of Kylin that can be deployed either in the cloud or on-premises. Some of the world’s largest financial institutes, e-commerce sites, telecom vendors, and consumer packaged goods companies have been using Kylin technology to solve their most challenging analytical problems.

In this article, we are going to take a look at the familiar notion of precomputation as a means of increasing analytics performance and to achieving sub-second response times for queries on extremely large (100’s of terabytes to petabytes) datasets in the cloud.

#cloud services #data #contributed #data-science

What is GEEK

Buddha Community

Multi-cloud Spending: 8 Tips To Lower Cost

A multi-cloud approach is nothing but leveraging two or more cloud platforms for meeting the various business requirements of an enterprise. The multi-cloud IT environment incorporates different clouds from multiple vendors and negates the dependence on a single public cloud service provider. Thus enterprises can choose specific services from multiple public clouds and reap the benefits of each.

Given its affordability and agility, most enterprises opt for a multi-cloud approach in cloud computing now. A 2018 survey on the public cloud services market points out that 81% of the respondents use services from two or more providers. Subsequently, the cloud computing services market has reported incredible growth in recent times. The worldwide public cloud services market is all set to reach $500 billion in the next four years, according to IDC.

By choosing multi-cloud solutions strategically, enterprises can optimize the benefits of cloud computing and aim for some key competitive advantages. They can avoid the lengthy and cumbersome processes involved in buying, installing and testing high-priced systems. The IaaS and PaaS solutions have become a windfall for the enterprise’s budget as it does not incur huge up-front capital expenditure.

However, cost optimization is still a challenge while facilitating a multi-cloud environment and a large number of enterprises end up overpaying with or without realizing it. The below-mentioned tips would help you ensure the money is spent wisely on cloud computing services.

• Deactivate underused or unattached resources

Most organizations tend to get wrong with simple things which turn out to be the root cause for needless spending and resource wastage. The first step to cost optimization in your cloud strategy is to identify underutilized resources that you have been paying for.

Enterprises often continue to pay for resources that have been purchased earlier but are no longer useful. Identifying such unused and unattached resources and deactivating it on a regular basis brings you one step closer to cost optimization. If needed, you can deploy automated cloud management tools that are largely helpful in providing the analytics needed to optimize the cloud spending and cut costs on an ongoing basis.

• Figure out idle instances

Another key cost optimization strategy is to identify the idle computing instances and consolidate them into fewer instances. An idle computing instance may require a CPU utilization level of 1-5%, but you may be billed by the service provider for 100% for the same instance.

Every enterprise will have such non-production instances that constitute unnecessary storage space and lead to overpaying. Re-evaluating your resource allocations regularly and removing unnecessary storage may help you save money significantly. Resource allocation is not only a matter of CPU and memory but also it is linked to the storage, network, and various other factors.

• Deploy monitoring mechanisms

The key to efficient cost reduction in cloud computing technology lies in proactive monitoring. A comprehensive view of the cloud usage helps enterprises to monitor and minimize unnecessary spending. You can make use of various mechanisms for monitoring computing demand.

For instance, you can use a heatmap to understand the highs and lows in computing visually. This heat map indicates the start and stop times which in turn lead to reduced costs. You can also deploy automated tools that help organizations to schedule instances to start and stop. By following a heatmap, you can understand whether it is safe to shut down servers on holidays or weekends.

#cloud computing services #all #hybrid cloud #cloud #multi-cloud strategy #cloud spend #multi-cloud spending #multi cloud adoption #why multi cloud #multi cloud trends #multi cloud companies #multi cloud research #multi cloud market

What are the benefits of cloud migration? Reasons you should migrate

The moving of applications, databases and other business elements from the local server to the cloud server called cloud migration. This article will deal with migration techniques, requirement and the benefits of cloud migration.

In simple terms, moving from local to the public cloud server is called cloud migration. Gartner says 17.5% revenue growth as promised in cloud migration and also has a forecast for 2022 as shown in the following image.

#cloud computing services #cloud migration #all #cloud #cloud migration strategy #enterprise cloud migration strategy #business benefits of cloud migration #key benefits of cloud migration #benefits of cloud migration #types of cloud migration

Cloud Analytics Migration: Go With The Need

The Cloud offers access to new analytics capabilities, tools, and ecosystems that can be harnessed quickly to test, pilot, and roll out new offerings.

The Cloud offers access to new analytics capabilities, tools, and ecosystems that can be harnessed quickly to test, pilot, and roll out new offerings. However, despite compelling imperatives, businesses are concerned as they move their analytics to the Cloud. Organizations are looking at service providers who can help them allocate resources and integrate business processes to boost performance, contain cost, and implement compliance across on-premise private and public cloud environments.

The most cited benefit of running analytics in the Cloud is increased agility. With computing resources and new tools available on-demand, analytics applications and infrastructure can be developed, deployed, and scaled up — or down — much more rapidly than can typically be done on-premises.

Unsurprisingly, cost reduction is seen as a significant benefit of cloud-based analytics. A complex algorithm processing large volumes of data may require thousands of CPUs and days of computing time, which can be prohibitive for companies without existing in-house compute and storage resources.

With the Cloud, organizations can rapidly access the required compute and storage power on demand and only pay for what they use. Research shows that migrating analytics to the Cloud can double an organization’s return on investment (ROI).

Standardization, cited as the third most crucial driver of migrating analytics to the Cloud, is strongly linked to the first two benefits of increased agility and reduced IT costs. Also, standardization helps organizations with simplified, streamlined IT management and shortened development cycles.

The Cloud offers access to new analytics capabilities, tools, and ecosystems that can be harnessed quickly to test, pilot, and roll out new offerings. For instance, organizations can take advantage of cloud-based data integration and preparation platforms with pre-built industry models. Leverage cloud services that offer powerful graphics processing unit (GPU)-based compute resources for complex analytics and tap into a collaborative ecosystem of data analysts within a federated data environment.

#big data #big data analytics #cloud migration #big data analytics platform #big data services #cloud analytics #big data solutions #big data analytics companies

Google Cloud: Caching Cloud Storage content with Cloud CDN

In this Lab, we will configure Cloud Content Delivery Network (Cloud CDN) for a Cloud Storage bucket and verify caching of an image. Cloud CDN uses Google’s globally distributed edge points of presence to cache HTTP(S) load-balanced content close to our users. Caching content at the edges of Google’s network provides faster delivery of content to our users while reducing serving costs.

For an up-to-date list of Google’s Cloud CDN cache sites, see https://cloud.google.com/cdn/docs/locations.

Task 1. Create and populate a Cloud Storage bucket

Cloud CDN content can originate from different types of backends:

• Compute Engine virtual machine (VM) instance groups
• Zonal network endpoint groups (NEGs)
• Internet network endpoint groups (NEGs), for endpoints that are outside of Google Cloud (also known as custom origins)
• Google Cloud Storage buckets

In this lab, we will configure a Cloud Storage bucket as the backend.

#google-cloud #google-cloud-platform #cloud #cloud storage #cloud cdn

Fractal Analytics Acquires Zerogons To Gear Up Cloud AI Business

Fractal Analytics Acquires AI solution provider Zerogons to strengthen Fractal’s Cloud AI business and accelerate the ‘data to decisions’ journey for its Fortune 500 clients.

Analytics solution provider, Fractal Analytics recently announced the acquisition of enterprise AI solutions platform for data scientists and data engineers, Zerogons. The amount of the acquisition is not disclosed and is stated as it will help strengthen Fractal’s Cloud AI business and accelerate the ‘data to decisions’ journey for its Fortune 500 clients.

#cloud ai #cloud ai business #fractal analytics #fractal cloud ai #cloud