Azure Data Factory: Management Hub

Management hub is a central location to access various high-level settings, actions, and configurations within Data Factory. Management hub can be accessed by clicking on the Manage tab on the Azure Portal UI.

Azure Data Factory: Manage tab

You can see various settings neatly grouped into categories in the screenshot above. Let’s have a look at these one by one:

Connections: Under this category you can manage various connection settings to external systems which will be used while creating a pipeline. In fact, the first step is making sure that we have the connections configured and tested properly.

  1. Linked Services: These are connections to external datastores, compute environments, filesystems etc. We have discussed Linked Services in a previous post.
  2. Integration Runtimes: An Integration Runtime is essentially, the compute resource which hosts the Data Factory. This enables the data factory to be compatible with various network configurations (including a hybrid cloud/on-prem infrastructure configuration). To know more about Integration Runtimes, please check out this previous post.
  3. Azure Purview: Purview is the unified data governance and data discovery service for Azure. To get a high-level understanding of Purview, please read previous post, Azure Purview Overview

Source Control: The settings under source control enable users to configure settings related to Git configuration (to enable CI/CD) and ARM templates. Let’s have a quick look at both:

  1. Git Configuration: All the Git-related settings including last published commit related information can be accessed under this tab. This will be discussed in more detail in a future post.
Azure Data Factory: Git Configuration Settings
  1. ARM template: ARM stands for Azure Resource Manager. ARM templates are JSON files that have configuration information (including compute infrastructure) for an Azure resource (such as a Data Factory). ARM templates can be used to migrate Azure resources to a new environment. This will be discussed in a future post.

Author: This category has settings related to the following:

  1. Triggers: These are used to schedule or automate execution of a data factory pipeline. To know more, please read the previous post here.
Azure Data Factory: Triggers
  1. Global Parameters: As the name suggests, this tab can be used to create parameters that can be accessed from multiple pipelines within the Data Factory. Check out this previous post, to learn more.

Security: The following security settings can be found under this tab:

  1. Customer Managed key: By default, Data Factory uses a randomly generated Microsoft-managed key to encrypt data at rest. But Data Factory also supports Bring-Your-Own-Key configuration (BYOK). Microsoft recommends creating the customer managed key using the Azure key vault. This will be discussed in more detail, in a future post.
  2. Credentials: We have discussed Service Principals and Managed Identities in a previous post about setting up Data Factory authentication to ADLS Gen2. This tab can be used to create user-assigned Service Principals and Managed Identities, which can then be used within linked services for authentication.

Reference: https://docs.microsoft.com/en-gb/azure/data-factory/author-management-hub

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: