Azure Data Lake Analytics and USQL

We have discussed Azure Data Lake Storage (ADLS) in a previous post. The main application of a Data Lake is to store various types of data at a low cost. But with the evolution of the Lakehouse architecture, it is becoming increasingly simple to analyse massive amounts of data and extract information cost effectively. To enable this, Microsoft has included an in-built service, called Azure Data Lake Analytics on the Azure cloud.

Azure Data Lake Analytics

Azure Data Lake Analytics is an analytics job service which is optimized for distributed data processing. It is designed to handle jobs of varied scale (from a few GBs to Peta Byte scale) and be able to scale and adapt the compute power as required, according to the job. This is achieved by dynamic provisioning of resources.

Let’s have a look at some of the main features of Azure Data Lake Analytics:

  1. USQL: Probably the biggest advantage of using Data Lake Analytics is that it uses a slightly modified version of T-SQL, called USQL. USQL integrates features from TSQL as well as C#, complementing and extending the programmability and data analysis capabilities of both.
  1. Visual Studio Integration: Seamless integration with Visual Studio provides users a familiar IDE to write, debug and test analytics jobs with ease.
  1. Compatibility with Azure Data Services: Data Lake Analytics can interact with ADLS Gen1, Azure SQL, Blob Storage and Synapse Analytics.
  1. Supports Diverse Workloads: Due to its scalability and flexibility, Data Lake Analytics can be used for various types of workloads such as, ETL, Machine Learning, Sentiment Analysis etc.
  1. Pay-per-job: Users are changed only for the time and resources that the job consumes (on a per-job run basis), which means no hardware or service specific charges.  

Limitations: One of the major limitations of Azure Data Lake Analytics is that it does not support ADLS Gen2 (only ADLS Gen1 support currently).

Reference: https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-overview

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: