Array Type Variable in Azure Data Factory - Real Time Example - Azure Data Factory Tutorial

Subject:  Array Type Variable in Azure Data Factory - Real Time Example - Azure Data Factory Tutorial.

In this article, we are going to learn about the Array type variable in the Azure data factory, so here in this article, we will go through some real-time examples to understand about Array type variable in the Azure data factory.

First of all open the Azure data factory studio, then go to the author tab then click the Pipelines tab then click on the + button to create a new Pipeline.

Then click on the variables tab then give the name to a variable, then select the type as we have three types of variables first one is Boolean, the second one is String and the third one is Array, so we will select Array, then give the default value, in my case it is [''Aisa'',''Europe'']. 

I have a file which is shown in the picture below, that contains records of more than 3 regions in which I need only Asia and Europe's data by using the variable. in the future, I might have hundreds of regions.

Now, go to the Azure data factory and find and drag the foreach loop activity in the working window, then go to the settings tab then click on add dynamic content.

Then click on the Vregion variable which we created before and click ok.

Then click on the pencil sign on the foreach activity and go inside the activity.

Inside the foreach activity, find and drag the copy activity and then go to the source tab and click on the + New button to create a new source dataset.

Select Azure SQL database and then click on continue.

Then it will ask for the linked service, click on + new to create a new linked service, Give the name, then select Azure subscription, then select server name, then select database name, then select authentication type, then provide username and password, then test the connection and then click on create.

Then Click on Query in the source tab and then click add dynamic content.

write the query and then click on ok.

Next, go to the sink tab and click on the + New button to create and new Sink dataset.

Select Azure blob storage and then click on continue.

Then select the CSV file as we need output data in csv file format, then click on continue.

Then it will ask for a linked service, click on + New to create a new linked service, Name the linked service, then select Azure subscription, then select storage account and then click on Create.

Then provide the file path then Select first row as header,'Optional', then Select import schema as  From Connection/store and then click Ok. 

In the sink tab click on open.

Then go to the parameters tab and then click on + New to create a new dataset parameter.

then go to the Connection tab and in the file click on Add dynamic content, then select the recently created parameter. 

Next, go back to pipeline and now we have to provide the value for this parameter, click on Add dynamic content and then select the value.

Next, go back to our Foreach activity and click on Debug, and then foreach loop activity will create 2 files and we provide the value. Once our debug process will be complete go to the blob storage and find the output files.

Video Demo: Array Type Variable in Azure Data Factory