Azure Data Factory has comprehensive data transformation and integration capabilities. For those looking for a simpler data integration experience compared to Data Factory Pipelines, Microsoft provides Mapping Data Flows which do not require writing any code.
Mapping Data Flows in Azure Data Factory are a visual way to design data transformation flows. The data flows created using Mapping Data Flows, internally become Azure Data Factory activities and get executed as a part of the Azure Data Factory Pipelines. The Data Flows are compatible with Azure Data Factory’s scheduling, control, flow, and monitoring features.
Mapping Data Flows can be created within the same Azure Data Factory resources pane as Pipelines.
On the next screen, there is a visual interface for designing Data Flows. The main feature on this page is the graph, which lets users choose the type of data transformation they would like to use:
Apart from the graphical user interface for creating Data Flows, there is a configuration panel at the bottom of the screen. Optimization Settings are available on the configuration panel with the options represented using pictures:
Mapping Data Flows become operational as Data Flow Activities within the Azure Data Factory. There is also a debug mode provided to view results of each transformation step for easy debugging.
Mapping Data Flows performance monitoring and other settings will be discussed in a future post.
Reference: Mapping data flows – Azure Data Factory | Microsoft Docs
One thought on “Mapping Data Flows in Azure Data Factory”