Azure Data Factory: System Variables

We have discussed about Global parameters in Azure Data Factory in a previous post. In today’s post, let’s learn about System variables in Azure Data Factory. System variables are built-in variables available to be used within every Data Factory Pipeline. The main purpose of having system variables is to make it easy for users to capture the commonly used, pipeline related information and pass these dynamically anywhere within the Pipeline. This information can come in handy while creating a log table to capture pipeline errors, while creating a logic based on the Pipeline trigger or for capturing the execution time of the pipeline.

Let’s see an example below. We have a Pipeline with a Lookup activity. One of the settings is to specify custom SQL query. But just below the box we have a link that says , Add dynamic content.

Azure Data Factory: Add dynamic content

Clicking on this, brings up a new blade where we can find all the available system variables.

Azure Data Factory: System Variables

Of course, this is just an example and within Data Factory, wherever we see the Add dynamic content option, we would be able to use the available System variables.

Most of the Data Factory System Variables are self-explanatory. I am listing them out below anyways:

System VariableDescriptionDatatype
@pipeline().DataFactoryName of the data or Synapse workspace the pipeline run is running inVarchar
@pipeline().PipelineName of the pipelineVarchar
@pipeline().RunIdID of the specific pipeline runGUID (varbinary)
@pipeline().TriggerTypeThe type of trigger that invoked the pipeline (for example, ScheduleTrigger, BlobEventsTrigger). For a list of supported trigger types, see Pipeline execution and triggers. A trigger type of Manual indicates that the pipeline was triggered manually.Varchar
@pipeline().TriggerIdID of the trigger that invoked the pipelineGUID (varbinary)
@pipeline().TriggerNameName of the trigger that invoked the pipelineVarchar
@pipeline().TriggerTimeTime of the trigger run that invoked the pipeline. This is the time at which the trigger actually fired to invoke the pipeline run, and it may differ slightly from the trigger’s scheduled time.Datetime
@pipeline().GroupIdID of the group to which pipeline run belongs.GUID (varbinary)
@pipeline()?TriggeredByPipelineNameName of the pipeline that triggers the pipeline run. Applicable when the pipeline run is triggered by an ExecutePipeline activity. Evaluate to Null when used in other circumstances. Note the question mark after @pipeline()Varchar
@pipeline()?TriggeredByPipelineRunIdRun ID of the pipeline that triggers the pipeline run. Applicable when the pipeline run is triggered by an ExecutePipeline activity. Evaluate to Null when used in other circumstances. Note the question mark after @pipeline()GUID (varbinary)
Azure Data Factory: System Variables (Pipeline Scope)

The table above only lists out the System variables in the Pipeline scope. For a list of all System Variables, please check the reference link below.

Reference: https://docs.microsoft.com/en-us/azure/data-factory/control-flow-system-variables

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: