Azure Data Factory Pipeline Trigger Executions

In the previous post, we looked at the manual execution of Azure Data Factory. The real benefit of Azure Data Factory is that pipelines can be automated. There are various ways to automate or trigger the execution of Azure Data Factory Pipelines:

Schedule Trigger: A trigger that invokes a pipeline execution at a fixed time or on a fixed schedule e.g. weekly, monthly etc.

Tumbling Window Trigger: Tumbling Window Triggers execute Azure Data Factory Pipeline at periodic time interval from a specified start time while retaining state. Time intervals for tumbling window are fixed size and there is no overlap between them.

Event Based Trigger: These triggers execute an Azure Data Factory Pipeline based on the occurrence of some event e.g. arrival or deletion of a new file in Azure Blob Storage.

It may be noted that both Schedule Trigger and Tumbling Window Trigger can be used to create recurring pipeline executions. The main difference between the two is that, the Tumbling Window Trigger waits for the pipeline execution to finish and captures the state of the pipeline execution. i.e. If the triggered pipeline run is cancelled, the corresponding tumbling window trigger run is marked cancelled. A Schedule Trigger, on the other hand, does not capture the state of the pipeline execution. It just marks the pipeline execution as successful as soon as the pipeline run starts, irrespective of the outcome of the pipeline execution.