How to Write Web Activity Output OR Azure Function Json Output to Azure Blob Storage - Azure Data Factory Tutorial

 Issue: How to Write Web Activity Output OR Azure Function Json Output to Azure Blob Storage - ADF Tutorial.

In this article, we are going to learn how to write web activity output or Azure function JSON output to Azure Blob storage, in this scenario we are going to get data from a website, that provides the free Dummy JSON data, so I have copied the web link and then get data from that link then write it to the blob storage. let's start our demonstration.

Open your Azure data factory studio, go to the Author tab, click on + sign to create a new pipeline, find and bring the Web activity, click on the settings tab, paste the copied web link, in the method select Get, as we are getting the data from this web link.


Click on the white canvas, go to the Variables tab, click on the + New button to create a variable, name your variable and leave the rest as it is.


Find and drag the set variable activity, connect with the web activity, go to the Variables tab, select the variable that we created, then in the value click on Add dynamic content and select the web activity then add the curly brackets in the expressions.


Find and bring the copy data activity, connect with the set variable activity, go to the source tab then click on the + New button to create a new source dataset.


Select Azure blob storage then click on continue.


Select the file format as CSV, then click on continue.


Name your dataset, select the linked service pointing to your blob storage, and then create a template file in your blob storage and then select that file, as we will write the data in that template file, for import schema select from connection/store, then click on ok.


In the Source tab click on the + New button and create an additional column, then provide the column name and in the values, click on ad dynamic content and select my variable. 


Once we are done with our source tab, go to the sink tab, click on the + New button to create a new sink dataset.


Select Azure blob storage then click on continue.


Select CSV as file format and click on continue.


Name your dataset, select the linked service, select the output folder path, import schemas will be from connection/store, then click on ok.


In the sink tab click on the Open button and provide the output file name, then in the escape character select No escape character, and in the Quote, character select No quote character. 




Next, go to the mapping tab, click on import schemas, it will bring the column in the template file, select the column and click on delete, leave the second column which we have created in the additional column, then we are good to go, click on Debug to execute the piepline.




Video Demo: How to Write Web Activity Output OR Azure Function Json Output to Azure Blob Storage - ADF Tutorial















 

5 comments:

  1. Pod system พอตไฟฟ้า smok ร้าน ขาย บุหรี่ไฟฟ้า ราคาถูกฟรี ครบวงจร ของแท้ 100% เพจจำหน่ายอุปกรณ์สำหรับ mod pod คือ pod1688 ส่งด่วน กทม 24 ชม. คอยล์

    ReplyDelete
  2. Wow, cool post. I'd like to write like this too - taking time and real hard work to make a great article... but I put things off too much and never seem to get started. Thanks though. Digital Agency Melbourne

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Very precious focus of course. Cloud computing services or applications and platforms are rapidly growing across all industries, serving as the IT infrastructure which drives new digital businesses. These type of platforms and applications have revolutionized the ways in which businesses function, and have made processes easier. A recent study proves that more than 78 percent of businesses today have at least some portion of their computing infrastructure in the cloud. Now, the technical details on how to write web activity output or Azure function JSON output to Azure Blob storage you elaborated was totally mind blowing and praiseworthy I think. All practical focuses were very noteworthy. The video tutorial was also very useful. I like the way you brought the activity output issue. Though my browsing intention was to learn about web design service in Edmonton area (I have already visited a professional service named up digital ca. their Six Step Web Design Services are really notable compared to the other services in the region. You can also check out this edmonton web design business), I was completely spell-bound with the knowledgeable details you explored here for us. Thank you very much for this great contribution and sharing online.

    ReplyDelete