In this article, we will show how to run an SSIS package using Azure Data factory.
In the previous articles, we showed how to copy data between different data stores, located in an on-premises server or in the cloud and how to perform data transformation on that data using the Azure Data Factory.
In this article, we will show how to run an SSIS package using the Azure Data Factory.
Azure SSIS IR is an Azure Data Factory fully managed cluster of virtual machines that are hosted in Azure and dedicated to run SSIS packages in the Data Factory, with the ability to scale up the SSIS IR nodes by configuring the node size and scale it out by configuring the number of nodes in the VMs cluster.
With Azure-SSIS IR, you can easily run the SSIS packages that are deployed into the SSIS catalog database, hosted in an Azure SQL Database server or an Azure SQL Database Managed Instance using the Project deployment model, or run the packages that are deployed into the file system, Azure Files, or SQL Server MSDB database that is hosted in an Azure SQL Database Managed Instance using the Package Deployment model.
In order to configure the Azure SSIS IR, open the Azure Data Factory using the Azure portal, then from the Overview page select the Author & Monitor option. From the Get Started window, you can configure the Azure SSIS IR from the Manage tab -> Integration Runtimes -> New Azure SSIS IR, or directly from the Configure SSIS Integration option, as shown below:
From the General Settings window of the Integration Runtime Setup, provide a meaningful name for the Azure SSIS IR, the region where this SSIS IR will be hosted, the size of the node in the integration runtime cluster, the number of nodes that will be assigned to the integration runtime cluster, whether to use Standard or Enterprise SQL Server edition for the integration runtime and if you plan to use your own SQL Server license to save money, as shown below:
From the Deployment settings window of Integration Runtime Setup, you need to specify whether to create SSISDB and deploy your packages into it, and/or use Azure-SSIS IR package stores to deploy the SSIS packages.
If you select to create an SSISDB, you will be asked to specify the Azure subscription and the region where the Azure SQL Server is located, taking into consideration that it is recommended to have both the Azure SQL Server and the Azure SSIS IR in the same region.
Also, you need to provide the endpoint of the Azure SQL Database server where SSISDB will be created, the SQL authentication or Azure AD authentication method and credentials that will be used by the Azure Data Factory to connect to the Azure SQL Server and the service tier for that Azure SQL Database server.
The Azure SSIS IR Package Store option provides you with the ability to manage the SSIS packages that are deployed into MSDB, file system, or Azure Files.
For the selected Azure SQL Database server, make sure that the Allow access to Azure services firewall setting is enabled, and that the server does not have an SSISDB instance already created, as using an existing SSISDB instance is not supported, as shown below:
On the Advanced settings window of Integration Runtime Setup, you will be asked to provide the maximum number of packages that will run concurrently per node in the integration runtime cluster, whether to add standard/express custom setups on your Azure-SSIS IR, whether to join the Azure SSIS IR to a VNET, and whether to configure a self-hosted IR as a proxy for the Azure-SSIS IR, as shown below:
This article will show Lift and Shift SSIS packages to Azure using Azure Data Factory V2.
Your Data Architecture: Simple Best Practices for Your Data Strategy. Don't miss this helpful article.
Azure Data Factory itself does not store any data. It lets you create data-driven workflows to orchestrate the movement of data between supported data stores. It also lets you process data by using compute services in other regions or in an on-premises environment.
In this article, we will show how we can use the Azure Data Factory to transform a specific data set.
In this post, we'll learn Getting Started With Data Lakes.<br><br> This Refcard dives into how a data lake helps tackle these challenges at both ends — from its enhanced architecture that's designed for efficient data ingestion, storage, and management to its advanced analytics functionality and performance flexibility. You'll also explore key benefits and common use cases.