Azure Data Factory: Filter Activity

In the previous post, we discussed the Switch Activity, which is useful for branching the control flow based on some condition. We will discuss about the Filter Activity in this post.

The purpose of Filter Activity is to process array items based on some condition. Consider a scenario where we would like to set the value of a variable to the current array item that satisfies some business rule or condition. We can use Filter activity to design the control flow.

Let’s have a look at the example below:

Azure Data Factory: Filter Activity Example

As we can see in the screenshot above, the pipeline contains two activities (Filter and ForEach) with an input array which is being passed on as the inputs pipeline parameter.

Based on condition under the settings tab, the Filter Activity only selects the array items which are greater than 3:

Azure Data Factory: Filter Activity Settings

For every array item greater than 3, the ForEach activity runs the Set Variable activity which converts the current filtered item value to string datatype and assigns it to the variable test.

Azure Data Factory: Filter Activity Example- Set Variable

As we can see from the above example, the Filter activity can be very useful while working with arrays. Without a specific activity to process array items, this would probably have to be written as SQL Code.

To learn more about ForEach activity, have a look at the previous post.

Reference: https://docs.microsoft.com/en-us/azure/data-factory/control-flow-filter-activity

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: