Migrating SQL workloads to Microsoft Azure: Planning the jump

In this article, we will discuss several points that should be considered when planning to migrate the on-premises SQL workload to Microsoft Azure cloud services. This article is the first step in a series of articles that discuss how to perform the SQL and No-SQL workload migration smoothly to the cloud.

Why migrate?

Data is one of the most precious “assets” in each company that drives business success. And as a proactive Data Engineer in an international company, you will always think how to secure your data at rest and in transit, and use the most optimal data platform technologies to serve the data to the application clients from any point in the earth as fast as possible with the minimum downtime or data loss possibilities.

With the growth of the company business, it is your responsibility to keep track of the data storage and retrieval speed in order not to lose your clients. Put yourself in the shoes of a client who is trying to submit an online order to buy from your online store, but the site is taking a long time to refresh the content and submit the order. For me, I will close the site and buy it from the nearest store!

If the Infrastructure administrator starts complaining about the limitation in the remaining resources in the current hosting machine, or the delay in receiving the new purchased resources, it is the suitable time to discuss with the management the choice to move your SQL workload to Microsoft Azure.

Initial Study

Before you think to send a meeting request to your management to discuss your idea about migrating the current SQL workload to Microsoft Azure, you need to take into consideration that it is not only one word that you need to mention to the management or a step by step tutorial that you can follow to perform the migration process. You should be prepared and ready for any question by preparing a comprehensive study that includes the current site problems and limitations, a plan for the design and implementation phases of the migration process, and the benefits that the company will gain from moving that workload to Azure from all performance, business growth handling and cost.

The initial study for the migration process should include, but may extend:

• The current SQL and No-SQL workload types in your company, such as OLTP and OLAP workloads
• The database administration and monitoring tools that you are using in the on-prems site
• The list of on-prems database engine types, versions, and locations
• The current database resources size and the expected resources growth ratio for each database
• The dependencies between the databases. This helps in selecting the list of candidate databases that will participate in the first migration wave and the database consolidation possibilities to reduce the cloud hosting cost
• The dependencies between the databases and the applications and how these applications interact with the databases. This will end up grouping the databases based on its dependencies. It is better to have a discussion with the development team to identify the criticality of each database for the business and see if it is possible to migrate the database applications to Microsoft Azure
• The different security and encryption requirements for each database
• The backup strategy and tools used for each database
• The list of all additional components that are involved in the data models, such as SSIS, SSAS, and SSRS that should be migrated
• The list of issues that you are facing in the current site, such as performance or availability
• The limitations in the current site, such as hardware upgrade limitation or unsupported features
• The Availability requirements for your databases
• The confirmed Restore Time Objective (RTO) and Restore Point Objective (RPO) for your databases and how they are met in the current site

Cloud Considerations

After checking the current situation, you should have a look at the cloud solutions that can be used to replace the current on-premises site. This includes learning the features available in each service and the pros and cons of each service in order to identify which service meets your requirements. With the different available cloud providers, I will concentrate on the Microsoft Azure database services in my articles, such as SQL on Azure VM, Azure SQL Database, Azure SQL Managed Instance, and Azure SQL Data Warehouse, and how to use it as a replacement for the on-premises ones.

Once you review the Microsoft Azure database-related services, your need to take into consideration the following points in your migration plan that may extend:

• Choose the suitable target data platform in the cloud, Infrastructure as a Services (IaaS) or Platform as a Service (PaaS), based on your technical requirements, and cost plan. This choice specifies what Azure services can be used and the control level on these services
• The Microsoft Azure services that can be used as a replacement for every single feature, component, or functionality in the on-prems site to serve your workloads
• The proper size for the Microsoft Azure services to fit the current site growth ratio with the minimal possible cost. Scaling up/down plans or scaling out/in plans at that stage will be a great step ahead
• The suitable region(s) to host each database that provides the lowest latency, based on the applications and client’s locations
• The Availability solution that should be used in Microsoft Azure, based on the selected Azure service, to meet your database availability requirements
• The cloud services configurations and features that can be used to meet the confirmed Restore Time Objective (RTO) and Restore Point Objective (RPO)
• The ability to achieve the security and privacy compliance and regulatory requirements of your organization using the cloud services
• The changes that should be performed on the current workload to be compatible with the target data platform technologies in Microsoft Azure and what transformation tools can be used to achieve that
• The new features in the cloud that can be used to optimize the current workload
• The administration and monitoring tools that can be used to replace the on-prems tools
• The ability to perform the security and encryption requirements for each database
• The backup and recovery cloud solutions can be used to design and automate the database backup strategy
• Any potential blockers for the migration process
• The tools that will be used to migrate the current workload to the cloud with minimal possible downtime and data loss
• Validation test strategy by migrating sample workload and how to measure the gains and compare it with the on-premises site
• The breaking fixes that should be performed after the migration process
• The rollback plan in case of any single possible failure

Now you can send the invitation to your management and discuss with them if migrating the current on-premises workload to the cloud is feasible, or you need to move with upgrading the current on-premises data center to handle the workload growth, serving the clients with highest possible availability and minimal data loss, without losing the company clients or having them frustrated from the company services.

#azure #migration #sql #sql-azure #sql-server #microsoft-azure

What is GEEK

Buddha Community

Introduction to Structured Query Language SQL pdf

SQL stands for Structured Query Language. SQL is a scripting language expected to store, control, and inquiry information put away in social databases. The main manifestation of SQL showed up in 1974, when a gathering in IBM built up the principal model of a social database. The primary business social database was discharged by Relational Software later turning out to be Oracle.

Models for SQL exist. In any case, the SQL that can be utilized on every last one of the major RDBMS today is in various flavors. This is because of two reasons:

1. The SQL order standard is genuinely intricate, and it isn’t handy to actualize the whole standard.

2. Every database seller needs an approach to separate its item from others.

Right now, contrasts are noted where fitting.

#programming books #beginning sql pdf #commands sql #download free sql full book pdf #introduction to sql pdf #introduction to sql ppt #introduction to sql #practical sql pdf #sql commands pdf with examples free download #sql commands #sql free bool download #sql guide #sql language #sql pdf #sql ppt #sql programming language #sql tutorial for beginners #sql tutorial pdf #sql #structured query language pdf #structured query language ppt #structured query language

Migrating SQL workloads to Microsoft Azure: Planning the jump

In this article, we will discuss several points that should be considered when planning to migrate the on-premises SQL workload to Microsoft Azure cloud services. This article is the first step in a series of articles that discuss how to perform the SQL and No-SQL workload migration smoothly to the cloud.

Why migrate?

Data is one of the most precious “assets” in each company that drives business success. And as a proactive Data Engineer in an international company, you will always think how to secure your data at rest and in transit, and use the most optimal data platform technologies to serve the data to the application clients from any point in the earth as fast as possible with the minimum downtime or data loss possibilities.

With the growth of the company business, it is your responsibility to keep track of the data storage and retrieval speed in order not to lose your clients. Put yourself in the shoes of a client who is trying to submit an online order to buy from your online store, but the site is taking a long time to refresh the content and submit the order. For me, I will close the site and buy it from the nearest store!

If the Infrastructure administrator starts complaining about the limitation in the remaining resources in the current hosting machine, or the delay in receiving the new purchased resources, it is the suitable time to discuss with the management the choice to move your SQL workload to Microsoft Azure.

Initial Study

Before you think to send a meeting request to your management to discuss your idea about migrating the current SQL workload to Microsoft Azure, you need to take into consideration that it is not only one word that you need to mention to the management or a step by step tutorial that you can follow to perform the migration process. You should be prepared and ready for any question by preparing a comprehensive study that includes the current site problems and limitations, a plan for the design and implementation phases of the migration process, and the benefits that the company will gain from moving that workload to Azure from all performance, business growth handling and cost.

The initial study for the migration process should include, but may extend:

• The current SQL and No-SQL workload types in your company, such as OLTP and OLAP workloads
• The database administration and monitoring tools that you are using in the on-prems site
• The list of on-prems database engine types, versions, and locations
• The current database resources size and the expected resources growth ratio for each database
• The dependencies between the databases. This helps in selecting the list of candidate databases that will participate in the first migration wave and the database consolidation possibilities to reduce the cloud hosting cost
• The dependencies between the databases and the applications and how these applications interact with the databases. This will end up grouping the databases based on its dependencies. It is better to have a discussion with the development team to identify the criticality of each database for the business and see if it is possible to migrate the database applications to Microsoft Azure
• The different security and encryption requirements for each database
• The backup strategy and tools used for each database
• The list of all additional components that are involved in the data models, such as SSIS, SSAS, and SSRS that should be migrated
• The list of issues that you are facing in the current site, such as performance or availability
• The limitations in the current site, such as hardware upgrade limitation or unsupported features
• The Availability requirements for your databases
• The confirmed Restore Time Objective (RTO) and Restore Point Objective (RPO) for your databases and how they are met in the current site

Cloud Considerations

After checking the current situation, you should have a look at the cloud solutions that can be used to replace the current on-premises site. This includes learning the features available in each service and the pros and cons of each service in order to identify which service meets your requirements. With the different available cloud providers, I will concentrate on the Microsoft Azure database services in my articles, such as SQL on Azure VM, Azure SQL Database, Azure SQL Managed Instance, and Azure SQL Data Warehouse, and how to use it as a replacement for the on-premises ones.

Once you review the Microsoft Azure database-related services, your need to take into consideration the following points in your migration plan that may extend:

• Choose the suitable target data platform in the cloud, Infrastructure as a Services (IaaS) or Platform as a Service (PaaS), based on your technical requirements, and cost plan. This choice specifies what Azure services can be used and the control level on these services
• The Microsoft Azure services that can be used as a replacement for every single feature, component, or functionality in the on-prems site to serve your workloads
• The proper size for the Microsoft Azure services to fit the current site growth ratio with the minimal possible cost. Scaling up/down plans or scaling out/in plans at that stage will be a great step ahead
• The suitable region(s) to host each database that provides the lowest latency, based on the applications and client’s locations
• The Availability solution that should be used in Microsoft Azure, based on the selected Azure service, to meet your database availability requirements
• The cloud services configurations and features that can be used to meet the confirmed Restore Time Objective (RTO) and Restore Point Objective (RPO)
• The ability to achieve the security and privacy compliance and regulatory requirements of your organization using the cloud services
• The changes that should be performed on the current workload to be compatible with the target data platform technologies in Microsoft Azure and what transformation tools can be used to achieve that
• The new features in the cloud that can be used to optimize the current workload
• The administration and monitoring tools that can be used to replace the on-prems tools
• The ability to perform the security and encryption requirements for each database
• The backup and recovery cloud solutions can be used to design and automate the database backup strategy
• Any potential blockers for the migration process
• The tools that will be used to migrate the current workload to the cloud with minimal possible downtime and data loss
• Validation test strategy by migrating sample workload and how to measure the gains and compare it with the on-premises site
• The breaking fixes that should be performed after the migration process
• The rollback plan in case of any single possible failure

Now you can send the invitation to your management and discuss with them if migrating the current on-premises workload to the cloud is feasible, or you need to move with upgrading the current on-premises data center to handle the workload growth, serving the clients with highest possible availability and minimal data loss, without losing the company clients or having them frustrated from the company services.

#azure #migration #sql #sql-azure #sql-server #microsoft-azure

Migrating SQL workloads to Microsoft Azure: Services Selection

In the previous article, Migrating SQL workloads to Microsoft Azure: Planning the jump, we discussed the main points that should be checked and considered while drawing your plan to migrate the SQL workload from the on-premises datacenters to Microsoft Azure. In this article, we will go through the different database services that are provided by Microsoft Azure to help you in selecting the proper service that can serve your SQL workload when migrating it to Microsoft Azure.

IaaS or PaaS

Before choosing the suitable Microsoft Azure database service that meets your requirements, you need to specify the suitable Azure platform. Microsoft Azure provides two high-level platform options: Infrastructure as a Services, also known as IaaS and Platform as a Service, also known as PaaS. The platform choice specifies the Azure services that can be used and the control that you can have over the services under that platform.

Choosing the IaaS platform, you are renting the IT infrastructure servers and virtual machines from the cloud provider. This includes storage, networks, and operating systems. With this platform, you are still responsible for and have control over the Operating System layer and all layers over the OS, including the installation of the services, the operating system patching, and so on.

On the other hand, the PaaS platform provides you with the ability of building, testing, and deploying your applications without worrying about the underlying infrastructure management. In other words, you are not responsible for installing an operating system or patching the machine with the latest security and system updates.

The following image shows your responsibilities, in light blue, and the list of layers that you don’t need to worry about, in dark blue, where the cloud service provider, Microsoft for example, is responsible for managing the tasks fall under that layer. You can see that you are responsible for everything when hosting your databases in your datacenter, requiring multiple teams to handle these tasks, which is not possible for the start-up and small companies, as shown below:

Microsoft Azure Database Services

Now we are familiar with the difference between the platforms provided by Microsoft Azure. We need to identify the database services that are provided under each platform.

In the IaaS platform, you can rent a virtual machine and install your SQL Server instance in that machine, where you will be responsible for the Operating System and SQL Server installation and administration tasks under that service.

Moving to PaaS, you can see that Microsoft Azure provides you with different choices based on your workload type. For example, you can use Azure SQL Database or Azure SQL Managed Instance for your transactional SQL workload and use Azure Cosmos DB for your No-SQL transactional workload. For the analytical workload, you can use the Azure SQL Data Warehouse instance, under the Azure Synapse Analytics service.

Let us discuss each SQL database service provided by Microsoft Azure briefly.

SQL Server on Azure VM

The IaaS platform provides you with the ability to install and run your SQL Server instance in a fully managed Azure virtual machine. This option is the best choice when you plan to perform a lift-and-shift from your on-prem environment to Microsoft Azure with the minimal possible changes on your applications and databases schema, providing you with full control over the SQL Server instance and the Operating System management and security configurations, allowing you to host any number of user databases on that SQL Server VM, and provide you with the ability to configure customized high availability and disaster recovery solution.

SQL Server on Azure VM is suitable for you if your company already has IT teams to administrate that virtual machine from OS, networking, and security perspectives. And you will be billed for both the storage used to store your data and the compute operations consumed on that VM.

Rather than waiting for the purchase approval for the new hardware, you can easily, in a few minutes, deploy a new virtual machine in Azure, install a new SQL Server instance using your own license and connect to that SQL Server instance, with the ability to scale it up and down based on your requirements, and stop it during the idle time and resume it again when needed.

Azure SQL Database

Azure SQL Database, categorized under the PaaS platform, is a cloud-computing database service that provides you with the ability to host and use SQL databases in the cloud without worrying about the hardware and the software requirements. Although you are not responsible for the hardware security, the operating system patching and security, and the database files, that are encrypted at rest using the TDE feature, you are still responsible for preventing unauthorized access to the data by limiting the allowed IP addresses from the firewall side and the authorized users from the database access and permissions configuration.

Azure SQL Database provides us with many features, including the ability to automate the backup operation and keep your backup for up to 10 years, create a readable secondary replica to distribute the reporting workload to another datacenter, tune the performance automatically, Point-In-Time Restore, built-in high-availability, and the ability to scale the database resources on the fly up and down, by changing the Database Throughput Unit (DTU) value, and scale-out with no downtime, without the need to wait for any new hardware purchase order as in the on-prems scaling processes, as shown below:

By providing the name of the database and a few other options, your database will be up and running and ready to serve your transactional workload in a few minutes. With no hardware or operating system to buy or manage, you will pay only for what you use. Feel free to use the Azure Total Cost of Ownership Calculator to estimate the cost of your PaaS service usage.

Azure SQL Database can be deployed as a single database, purchased by DTU or vCore models, with its own set of resources managed by a logical SQL Server, that can be used when the database usage is stable. It can be also deployed using an elastic pool, purchased by eDTU or vCore models, that contains a group of databases that share the same set of resources and managed by a logical SQL Server, providing the best choice for the databases with frequently changing usage patterns. If your application surface area scoped at the database level, using the Azure SQL Database is the best choice.

#azure #migration #sql #sql-server #sql-azure #microsoft-azure

How to set up Azure Data Sync between Azure SQL databases and on-premises SQL Server

In this article, you learn how to set up Azure Data Sync services. In addition, you will also learn how to create and set up a data sync group between Azure SQL database and on-premises SQL Server.

In this article, you will see:

• Overview of Azure SQL Data Sync feature
• Discuss key components
• Comparison between Azure SQL Data sync with the other Azure Data option
• Setup Azure SQL Data Sync
• More…

Azure Data Sync

Azure Data Sync —a synchronization service set up on an Azure SQL Database. This service synchronizes the data across multiple SQL databases. You can set up bi-directional data synchronization where data ingest and egest process happens between the SQL databases—It can be between Azure SQL database and on-premises and/or within the cloud Azure SQL database. At this moment, the only limitation is that it will not support Azure SQL Managed Instance.

#azure #sql azure #azure sql #azure data sync #azure sql #sql server

Creating and Cataloging SQL pools in Azure SQL Server

This article will walk you through creating a new SQL pool within an existing Azure SQL Server as well as catalog the same using the Azure Purview service.

Introduction

Data is generated by transactional systems and typically stored in relational data repositories. This data is generally used by live applications and for operational reporting. As this data volume grows, this data is often required by other analytical repositories and data warehouses where it can be used for referential purposes and adding more context to other data from across the organization. Transactional systems (also known as Online Transaction Processing (OLTP) systems) usually need a relational database engine, while analytical systems (also known as Online Analytical Processing (OLAP) systems) usually need analytical data processing engines. On Azure cloud, it is usually known that for OLTP requirements, SQL Server or Azure SQL Database can be employed, and for analytical data processing needs, Azure Synapse and other similar services can be employed. SQL Pools in Azure Synapse host the data on an SQL Server environment that can process the data in a massively parallel processing model, and the address of this environment is generally the name of the Azure Synapse workspace environment. At times, when one has already an Azure SQL Server in production or in use, the need is to have these SQL Pools on an existing Azure SQL Server instance, so data in these SQL pools can be processed per the requirements on an OLAP system as well as the data can be co-located with data generated by OLTP systems. This can be done by creating SQL Pools within the Azure SQL Server instance itself. In this article, we will learn to create a new SQL Pool within an existing Azure SQL Server followed by cataloging the same using the Azure Purview service.

Pre-requisite

As we intend to create a new SQL Pool in an existing Azure SQL Server instance, we need to have an instance of Azure SQL in place. Navigate to Azure Portal, search for Azure SQL and create a new instance of it. We can create an instance with the most basic configuration for demonstration purposes. Once the instance is created, we can navigate to the dashboard page of the instance and it would look as shown below.

As we are going to catalog the data in the dedicated SQL Pool hosted on Azure SQL instance, we also need to create an instance of Azure Purview. We would be using the Azure Purview studio from the dashboard of this instance, tonregister this SQL Pool as the source and catalog the instance.

#azure #sql azure #azure sql server #sql #sql #azure