We have discussed about Global parameters in Azure Data Factory in a previous post. In today’s post, let’s learn about System variables in Azure Data Factory. System variables are built-in variables available to be used within every Data Factory Pipeline. The main purpose of having system variables is to make it easy for users to capture the commonly used, pipeline related information and pass these dynamically anywhere within the Pipeline. This information can come in handy while creating a log table to capture pipeline errors, while creating a logic based on the Pipeline trigger or for capturing the execution time of the pipeline.
Let’s see an example below. We have a Pipeline with a Lookup activity. One of the settings is to specify custom SQL query. But just below the box we have a link that says , Add dynamic content.
Clicking on this, brings up a new blade where we can find all the available system variables.
Of course, this is just an example and within Data Factory, wherever we see the Add dynamic content option, we would be able to use the available System variables.
Most of the Data Factory System Variables are self-explanatory. I am listing them out below anyways:
|@pipeline().DataFactory||Name of the data or Synapse workspace the pipeline run is running in||Varchar|
|@pipeline().Pipeline||Name of the pipeline||Varchar|
|@pipeline().RunId||ID of the specific pipeline run||GUID (varbinary)|
|@pipeline().TriggerType||The type of trigger that invoked the pipeline (for example, ScheduleTrigger, BlobEventsTrigger). For a list of supported trigger types, see Pipeline execution and triggers. A trigger type of Manual indicates that the pipeline was triggered manually.||Varchar|
|@pipeline().TriggerId||ID of the trigger that invoked the pipeline||GUID (varbinary)|
|@pipeline().TriggerName||Name of the trigger that invoked the pipeline||Varchar|
|@pipeline().TriggerTime||Time of the trigger run that invoked the pipeline. This is the time at which the trigger actually fired to invoke the pipeline run, and it may differ slightly from the trigger’s scheduled time.||Datetime|
|@pipeline().GroupId||ID of the group to which pipeline run belongs.||GUID (varbinary)|
|@pipeline()?TriggeredByPipelineName||Name of the pipeline that triggers the pipeline run. Applicable when the pipeline run is triggered by an ExecutePipeline activity. Evaluate to Null when used in other circumstances. Note the question mark after @pipeline()||Varchar|
|@pipeline()?TriggeredByPipelineRunId||Run ID of the pipeline that triggers the pipeline run. Applicable when the pipeline run is triggered by an ExecutePipeline activity. Evaluate to Null when used in other circumstances. Note the question mark after @pipeline()||GUID (varbinary)|
The table above only lists out the System variables in the Pipeline scope. For a list of all System Variables, please check the reference link below.