Category Archives: Uncategorized

We have discussed Managed Identity authentication for Azure Data Factory in a previous post (which was about setting up DB permissions for a linked service). Managed Identity is one of the security principals used to manage access on the Azure cloud. Azure provides a comprehensive access control system for managing access using in-built roles. ThisContinue reading “How does Azure Role Based Access Control (RBAC) work?”

We have discussed various ways to setup authentication for Azure Data Factory Linked Service to ADLS Gen2, in previous post. One of the standard authentication methods is System-Managed Identity. While setting up proper roles on Azure Portal is enough for this scenario, lets have a look at another scenario where we are using Managed IdentityContinue reading “How to setup DB permissions for Azure Data Factory (Managed Identity)?”

We have discussed Azure Data Catalog briefly in previous post about Azure Data Services. In this post, let’s have a look at a recently launched similar service, Azure Purview. Azure Purview is Microsoft’s new unified data governance and data discovery platform. It allows users to manage an organization-wide map of various data sources, including dataContinue reading “Azure Purview Overview”

The most convenient way to move data to the cloud for analytics use cases, is to create a Data Lake using ADLS. Azure Data Factory (or Synapse Analytics) Pipelines can then be used to transform this data for further analysis. This can be done by creating a Linked Service. We have discussed about linked servicesContinue reading “Configure Azure Data Factory Authentication to ADLS Gen2”

One of the crucial aspects of any data integration project with Azure Data Factory is data security. With increasing focus on data security, preventing unauthorised access to the data stores is on every organization’s information security goals. In this post, lets look at some data access strategies that Data Stores on the Azure cloud provide,Continue reading “Azure Data Factory: Data Access Strategies”

One of the ways to secure data stored in Synapse Analytics Dedicated SQL Pool, is to enable Transparent Data Encryption (TDE). Enabling TDE ensures that all data stored in the dedicated SQL pool remains encrypted while it is at rest (i.e., not being accessed or in transit.) In this post, we will see how weContinue reading “How to manage encryption in Synapse Analytics Dedicated SQL Pool?”

In the previous post, we discussed Lakehouse, had a brief look at the DW architecture and how it has evolved with time. In this post, lets have a look at how we can use Microsoft Azure services and features to implement a Lakehouse. In case you are not familiar with DW architecture evolution, a LakehouseContinue reading “What is Delta Lake?”

With the evolution of scalable cloud technology and exponential growth of digital technologies, the preferred Data Warehouse (DW) architecture is going through a radical change. Data Lakehouse is an evolution of the DW architecture in response to the current digital environment. In order to fully appreciate how we got here, lets have a brief lookContinue reading “What is a (Data) Lakehouse?”

IDENTITY is a SQL keyword to specify that a column in the table will be used as a Surrogate Key. Surrogate Key column contains numerical data to uniquely identify a row in the table. A surrogate key is used to assign unique row identifiers in various scenarios. One of the most common use cases forContinue reading “Comparing SQL Server and Synapse Analytics IDENTITY Columns”

Synapse Analytics Dedicated SQL Pools support a new concept called query labels. Query labels are tags that can be applied to SQL queries using the OPTION keyword. e.g. As we know, Synapse Analytics has an MPP architecture where data is stored across multiple nodes. This means that table rows can be distributed across multiple nodesContinue reading “Query Labels in Azure Synapse Analytics Dedicated SQL Pool”