Azure SQL | Read Data Lake Files using Synapse SQL External Tables

Azure SQL | Read Data Lake Files using Synapse SQL External Tables

Azure SQL can read Azure Data Lake storage files using Synapse SQL external tables. Synapse SQL enables you to implement T-SQL functionalities in Azure SQL that read the content of CSV, PARQUET, and JSON files that are placed on Azure Data Lake storage.

There are many scenarios where you might need to access external data placed on Azure Data Lake from your Azure SQL database. Some of your data might be permanently stored on the external storage, you might need to load external data into the database tables, etc.

Azure SQL supports the OPENROWSET function that can read CSV files directly from Azure Blob storage. This function can cover many external data access scenarios, but it has some functional limitations.

You might also leverage an interesting alternative –  serverless SQL pools in the Azure Synapse Analytics. In this article, I will explain how to leverage a serverless Synapse SQL pool as a bridge between Azure SQL and Azure Data Lake storage.

This article applies both on Azure SQL database and Azure SQL managed instance. External tables with type RDBMS can be used in any Azure SQL flavor. Just note that External tables are still in public preview.

Let us first see what is Synapse SQL pool and how can be used from Azure SQL.

What is a serverless Synapse SQL pool?

A  serverless Synapse SQL pool is one of the components of the Azure Synapse Analytics workspace. It is a service that enables you to query files on the Azure storage. You can access the Azure Data Lake files using the T-SQL language that you are using in Azure SQL.

azure sql big data azuresql azuresqldb big data data analytics serverless synapse

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Support for Synapse SQL serverless in Azure Synapse Link for Azure Cosmos DB

Co-authored by Rodrigo Souza, Ramnandan Krishnamurthy, Anitha Adusumilli and Jovan Popovic (Azure Cosmos DB and Azure Synapse Analytics teams) Azure Synapse Link now supports querying Azure Cosmos DB data using Synapse SQL serverless. This capability, available in public preview, allows you to use familiar analytical T-SQL queries and build powerful near real-time BI dashboards on Azure Cosmos DB data.

Introduction to Structured Query Language SQL pdf

SQL stands for Structured Query Language. SQL is a scripting language expected to store, control, and inquiry information put away in social databases. The main manifestation of SQL showed up in 1974, when a gathering in IBM built up the principal model of a social database. The primary business social database was discharged by Relational Software later turning out to be Oracle.

Big Data Analytics: Unrefined Data to Smarter Business Insights - TopDevelopers.co

For Big Data Analytics, the challenges faced by businesses are unique and so will be the solution required to help access the full potential of Big Data.

Silly mistakes that can cost ‘Big’ in Big Data Analytics

‘Data is the new science. Big Data holds the key answers’ - Pat Gelsinger The biggest advantage that the enhancement of modern technology has brought

Big Data can be The ‘Big’ boon for The Modern Age Businesses

We need no rocket science in understanding that every business, irrespective of their size in the modern-day business world, needs data insights for its expansion. Big data analytics is essential when it comes to understanding the needs and wants of a significant section of the audience.