How to use Append Variable activity in Azure Data Factory - Azure Data Factory Tutorial 2022

Topic: How to use Append Variable activity in Azure Data Factory 


In this article we are going to learn how to use append variable activity in Azure Data Factory, we are going to perform a real-time example in which what we will do, we will be emailing the list of the files which you have processed in your data factory so we will concatenate all those lists of the files and then email. let's start our demonstration.

First of all, open your Azure Data Factory studio, go to the author tab, click on the + button to create a new pipeline, then find and drag the Get Metadata activity.


Rename the Get Metadata activity for your convenience, then go to the Dataset tab, click on the + New button to create a new dataset


 Select Azure Blob Storage and then click on Continue.


Select the File format and then click on continue.


Name your dataset, select the linked service if you have already created, or create a new one, provide the input file path, select the none as import schema as per your requirement, then click on ok.


Go to the dataset tab, and select the Child Items in Field list, as we are going to read the list of the files.

Find and drag the ForEach Loop Activity, connect with the Get Metadata activity, go to the settings tab then click on the Add dynamic content.


Select the Metadata activity, expressions will be shown in the above box, then add ".Childitems" and then click on Ok



Go to the variable tab, click on the  + button to create a new variable, name your variable, and select the type.


Go inside ForEach activity, bring the wait activity or copy data activity as per your scenario, then find and bring the append variable activity, connect both of them then go to the Variables tab, select the name and click on Add Dynamic content.


Click on ForEach, then the expression will be shown in the above box, then add ".name" and click on Ok.


Outside the ForEach activity, find and bring the Set variable, then connect with the ForEach, then create a string type variable and then click on the set variable activity, go to the Variables tab, select the variable and then click on Add dynamic content.



Click on the variable name which we have created earlier, then add the curly parentheses around and click on ok.


Next. go to the Azure portal, find and open the Logic Apps, then click on the + Add button to create a new Logic app, in the basics tab select your Azure subscription, select the resource group, select the logic app type, name your logic app, select your region then click on Review + create and then click on the create.



Once the logic app is created, Open the resource and here we will setup the Email part, search for the "Request", then click on the request and then select http request.


Once the HTTP request is received click on the Use sample payload to generate schema.


In this window enter the expressions and then click on Done.



Click on the Method then select Post then click on Next.
.

Select the Email service, in my case it is Gmail, then provide a connection name, select the authentication type and click on Sign In, it will redirect to a new window and then you have to provide your user name and password for the email account. 




Once you are signed in, it will redirect you to another window where you have to provide the email address where you want to send the email, then provide and subject and the message the select the List of files variable from the drop down window, and then finally click on Save.



Once you are done with the email part, click on the HTTP box and then copy the HTTP POST URL link.


Next, go to the Azure Data Factory studio, then find and drag the web activity, then connect with the set variable activity, go to the settings tab provide the HTTP URL link, select the method, provide the body, then publish the pipeline, and then debug your pipeline.






Video Demo: How to use Append Variable activity in Azure Data Factory






5 comments:

  1. This is a great to visit here I really like your informative post this post help me in my construction project management dissertation topics, I start following your blog for further informative content.

    ReplyDelete
  2. The Download Date of Tamil Nadu HSC Model Paper 2023 will be announced on the official website of the Tamil Nadu Board of Secondary Education in Chennai. Stop waiting for the TN 12th Question Paper 2023 to get out. TN Plus Two Blueprint 2023 You have arrived to the correct location to obtain your TN HSC Model Paper 2023. In March, TNBSE conducts examinations for the 12th HSC class now is the time to obtain the board exam Question Paper 2023 from the website. Students will be needed to enter their Syllabus and Subject in order to access the Question Paper 2023. They may also publish a copy of the preliminary previous paper.

    ReplyDelete
  3. A friend of mine recommended brillianthackers800@gmail.com to me although i was feeling skeptical about them at first but i went ahead to message them and they helped me settle bank loans and they also helped me increase my credit score you can chat them on WhatsApp +14106350697

    They will surely help you with your hacking problems at a very affordable price and they are very reliable and they deliver a perfect job

    ReplyDelete
  4. I am really very happy to visit your blog. Directly I am found which I truly need. please visit our website for more information
    Top 5 Data Extraction Tools

    ReplyDelete
  5. Brothers/Sister's. Are you looking for loan to finance your large or small business,we'll help you get the large amount of loan you desire for your business,we offer first class business and commercial loan to enable small scale business attain success in obtaining start up or refinance their business Bad credit rating accepted and poor business performance are accepted.apply today

    Are you a business man or woman? Do you need funds to start up your own business? Do you need loan to settle your debt or pay off your bills or start a nice business? Do you need funds to finance your project? We Offers guaranteed loan services of any amount and to any part of the world for (Individuals, Companies, Realtor and Corporate Bodies) at our superb interest rate of 3%. For application and more information send replies to the

    via email ronniefinancehome247@gmail.com
    Whats App +91 93118 56893
    LOAN APPLICATION FORM
    Name:
    Sex:
    Address:
    Occupation:
    State:
    Country:
    Phone Number:
    Loan Duration:
    Loan Needed:
    Have you applied before?:
    Email us ronniefinancehome247@gmail.com
    Whats-App +91 93118 56893
    Dr. Mark Thomas

    ReplyDelete