Azure SQL can read Azure Data Lake storage files using Synapse SQL external tables. Synapse SQL enables you to implement T-SQL functionalities in Azure SQL that read the content of CSV, PARQUET, and JSON files that are placed on Azure Data Lake storage.
There are many scenarios where you might need to access external data placed on Azure Data Lake from your Azure SQL database. Some of your data might be permanently stored on the external storage, you might need to load external data into the database tables, etc.
Azure SQL supports the OPENROWSET function that can read CSV files directly from Azure Blob storage. This function can cover many external data access scenarios, but it has some functional limitations.
You might also leverage an interesting alternative – serverless SQL pools in the Azure Synapse Analytics. In this article, I will explain how to leverage a serverless Synapse SQL pool as a bridge between Azure SQL and Azure Data Lake storage.
This article applies both on Azure SQL database and Azure SQL managed instance. External tables with type RDBMS can be used in any Azure SQL flavor. Just note that External tables are still in public preview.
Let us first see what is Synapse SQL pool and how can be used from Azure SQL.
A serverless Synapse SQL pool is one of the components of the Azure Synapse Analytics workspace. It is a service that enables you to query files on the Azure storage. You can access the Azure Data Lake files using the T-SQL language that you are using in Azure SQL.
Co-authored by Rodrigo Souza, Ramnandan Krishnamurthy, Anitha Adusumilli and Jovan Popovic (Azure Cosmos DB and Azure Synapse Analytics teams) Azure Synapse Link now supports querying Azure Cosmos DB data using Synapse SQL serverless. This capability, available in public preview, allows you to use familiar analytical T-SQL queries and build powerful near real-time BI dashboards on Azure Cosmos DB data.
SQL stands for Structured Query Language. SQL is a scripting language expected to store, control, and inquiry information put away in social databases. The main manifestation of SQL showed up in 1974, when a gathering in IBM built up the principal model of a social database. The primary business social database was discharged by Relational Software later turning out to be Oracle.
For Big Data Analytics, the challenges faced by businesses are unique and so will be the solution required to help access the full potential of Big Data.
‘Data is the new science. Big Data holds the key answers’ - Pat Gelsinger The biggest advantage that the enhancement of modern technology has brought
We need no rocket science in understanding that every business, irrespective of their size in the modern-day business world, needs data insights for its expansion. Big data analytics is essential when it comes to understanding the needs and wants of a significant section of the audience.