Topic: How to use Append Variable activity in Azure Data Factory
In this article we are going to learn how to use append variable activity in Azure Data Factory, we are going to perform a real-time example in which what we will do, we will be emailing the list of the files which you have processed in your data factory so we will concatenate all those lists of the files and then email. let's start our demonstration.
First of all, open your Azure Data Factory studio, go to the author tab, click on the + button to create a new pipeline, then find and drag the Get Metadata activity.
Rename the Get Metadata activity for your convenience, then go to the Dataset tab, click on the + New button to create a new dataset
Select Azure Blob Storage and then click on Continue.
Select the File format and then click on continue.
Name your dataset, select the linked service if you have already created, or create a new one, provide the input file path, select the none as import schema as per your requirement, then click on ok.
Go to the dataset tab, and select the Child Items in Field list, as we are going to read the list of the files.
Find and drag the ForEach Loop Activity, connect with the Get Metadata activity, go to the settings tab then click on the Add dynamic content.
Select the Metadata activity, expressions will be shown in the above box, then add ".Childitems" and then click on Ok
Go to the variable tab, click on the + button to create a new variable, name your variable, and select the type.
Go inside ForEach activity, bring the wait activity or copy data activity as per your scenario, then find and bring the append variable activity, connect both of them then go to the Variables tab, select the name and click on Add Dynamic content.
Click on ForEach, then the expression will be shown in the above box, then add ".name" and click on Ok.
Outside the ForEach activity, find and bring the Set variable, then connect with the ForEach, then create a string type variable and then click on the set variable activity, go to the Variables tab, select the variable and then click on Add dynamic content.
Click on the variable name which we have created earlier, then add the curly parentheses around and click on ok.
Next. go to the Azure portal, find and open the Logic Apps, then click on the + Add button to create a new Logic app, in the basics tab select your Azure subscription, select the resource group, select the logic app type, name your logic app, select your region then click on Review + create and then click on the create.
Once the HTTP request is received click on the Use sample payload to generate schema.
In this window enter the expressions and then click on Done.
Click on the Method then select Post then click on Next.
.
Select the Email service, in my case it is Gmail, then provide a connection name, select the authentication type and click on Sign In, it will redirect to a new window and then you have to provide your user name and password for the email account.
Once you are signed in, it will redirect you to another window where you have to provide the email address where you want to send the email, then provide and subject and the message the select the List of files variable from the drop down window, and then finally click on Save.
Once you are done with the email part, click on the HTTP box and then copy the HTTP POST URL link.
Next, go to the Azure Data Factory studio, then find and drag the web activity, then connect with the set variable activity, go to the settings tab provide the HTTP URL link, select the method, provide the body, then publish the pipeline, and then debug your pipeline.
