Azure Data Factory: Monitor Pipelines using Azure Monitor   

In the previous post, we discussed about creating an Elapsed Time metric to monitor long-running Pipelines in real time using Azure Monitor. In this post, lets discuss about Azure Monitor and how it can be used to monitor Data Factory Pipelines.

The easiest way to monitor Pipelines visually, is to go the Azure Monitor UI within the Data Factory Studio. To be able to do this, click on the Azure Monitor icon as shown in the screenshot below:

Let’s go through the tabs under Monitor:

  1. Dashboard: This tab gives you a graphical high-level status of the Pipelines, Activities and Triggers in the Data Factory:
  1. Pipeline Runs: This tab shows you the status of the individual Pipelines. We can further drill down to check the status of the underlying child Pipelines and Activities.

Please note the Triggered and Debug filters at the top. This means that this screen can also be used to monitor the Debug (Manual test) runs of the Pipelines.

  1. Trigger Runs: As the name suggests, this tab shows the status of the Triggers that have been created in the Data Factory:
  1. Integration Runtimes: This tab can be used to monitor the Integration Runtimes that are linked to the Data Factory.
  1. Data flow debug: If you have created any Dataflows as a part of your Pipelines, they can be monitored here.
  2. Alters & Metrics: Last but by no means the least, Alerts and Metrics tab lets you create Alert rules based on various Data Factory metrics captured by Azure Monitor.

One thought on “Azure Data Factory: Monitor Pipelines using Azure Monitor   

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: