Azure Data Factory: Fail Activity

During some scenarios in Azure Data Factory, we may want to intentionally stop the execution of the pipeline. An example could be when we want to check the existence of a file or folder using Get Metadata activity. We may want to fail the pipeline if the file/folder does not exist. To achieve this, weContinue reading “Azure Data Factory: Fail Activity”

Azure Data Factory: Filter Activity

In the previous post, we discussed the Switch Activity, which is useful for branching the control flow based on some condition. We will discuss about the Filter Activity in this post. The purpose of Filter Activity is to process array items based on some condition. Consider a scenario where we would like to set theContinue reading “Azure Data Factory: Filter Activity”

Azure Data Factory: Validation Activity

We looked at a specific scenario for branching the control flow based on some business rule or condition in the previous post. In this post, we are going to look at a different scenario. Sometimes, during a scheduled data load, the source data store may not be ready i.e., the files in the source folderContinue reading “Azure Data Factory: Validation Activity”

Azure Data Factory: Switch Activity

While designing ETL data flows, we will come across some scenarios where we must branch the control flow based on a business rule or conditional expression. The easiest way to implement conditional branching, in Data Factory is by using the Switch Activity. It is very easy to understand and implement if you are familiar withContinue reading “Azure Data Factory: Switch Activity”