Monthly Archives: November 2021

During some scenarios in Azure Data Factory, we may want to intentionally stop the execution of the pipeline. An example could be when we want to check the existence of a file or folder using Get Metadata activity. We may want to fail the pipeline if the file/folder does not exist. To achieve this, weContinue reading “Azure Data Factory: Fail Activity”

In one of the previous posts, we discussed how we can use Validation activity to design the Pipeline to wait for a scheduled time and retry. There is another way to introduce a delay in the Pipeline. Wait activity can be used to pause the execution of the Pipeline for a fixed amount of time.Continue reading “Azure Data Factory: Wait Activity”

In the previous post, we discussed the Switch Activity, which is useful for branching the control flow based on some condition. We will discuss about the Filter Activity in this post. The purpose of Filter Activity is to process array items based on some condition. Consider a scenario where we would like to set theContinue reading “Azure Data Factory: Filter Activity”

We looked at a specific scenario for branching the control flow based on some business rule or condition in the previous post. In this post, we are going to look at a different scenario. Sometimes, during a scheduled data load, the source data store may not be ready i.e., the files in the source folderContinue reading “Azure Data Factory: Validation Activity”

While designing ETL data flows, we will come across some scenarios where we must branch the control flow based on a business rule or conditional expression. The easiest way to implement conditional branching, in Data Factory is by using the Switch Activity. It is very easy to understand and implement if you are familiar withContinue reading “Azure Data Factory: Switch Activity”